Simulation & Cyber Exercises

Imagine the following scenario: you have just completed an exercise that took months for your team to plan. There is a collective sense of relief that the exercise went well, that the participants felt the exercise was worthwhile, and that all your exercise objectives were met. At the same time, you have the sense that the exercise lacked something, that it could have been more engaging. So you ask the participants during an after action review meeting if there’s anything you could have done to improve the exercise. There is a brief silence, then someone (probably one of the ex-military guys) has a suggestion: “Have you thought about using computer simulation?”

Every exercise is, of course, a simulation. Exercises indulge in a manufactured reality for the purpose of a collective, controlled thought experiment. At the heart of every “simulation”, whether it is a computer game or a weather forecast, has at its core a model. Computers are fast and accurate calculators, but computer simulations are only as good as the models on which they are based, and this leads to a pervasive schism.  
 
Reactions to computer simulations and the models they are based on tend to divide along sharp lines, toward two extremes that can be described as “mystical” and “cynical”. Simulation mystics tend to infuse computer models with magical powers, likening them to crystal balls with the ability to predict the future. Occupying the other side of the coin,  “cynics”, perhaps due to some negative experience with computer simulations that failed to accurately predict the future, tend to flee to the opposite extreme. Cynics declare (correctly) that models are limited due to complexity, assumptions, and limited data, and then (incorrectly) assert that such models have no value whatsoever.  
 
Of the two extremes the mystical interpretation is arguably the more troublesome, particularly in the context of exercises.  Invite someone to participate in an exercise and you’re likely to get an ambivalent response, but tell them that the exercise will involve computer simulation and your prospective participant’s eyes may grow wide with excitement as they imagine an arcade-like experience complete with immersive 3-d graphics. Even worse, they may anticipate that the exercise will reveal to them all of the consequences of high-impact events like terror attacks or natural disasters. Then try to beat back this mystical enthusiasm with a cool-headed explanation of uncertainty and the limits of your model, and watch as your mystic becomes a cynic before your eyes, or simply assumes that you must be too lazy to make your simulation “accurate”.  

The value of computer simulation comes not from prediction, but from transforming exercises from narrative experiences to opportunities for experimentation. Narrative exercises tend to reinforce our existing biases about the world, where experimental exercises allow participants to explore new ways of reacting to disasters. A computer model is like a black box, where different inputs yield different results, and a well-designed computer model will include second and third order effects that go beyond linear, cause-and-effect thinking. Experimentation encourages a scientific frame of mind, where all assumptions are challenged and familiar narratives can be broken in the safety of an exercise. Experimentation requires thinking and reflection, which are the common objective of all exercises.