Journal on Policy and Complex Systems
• Region and Tradeoff Plots . Emphasize charts showing how outcomes vary with combinations of causal-factor values and model structures . Instead of showing results and waiting for the recipient ’ s “ What if ?” questions , preemptively show results across the assumption space .
We illustrate this last approach in the next two figures . The top pane in Figure 3 shows a conventional comparison of two options for some standard case . The lower pane is a region plot that is much richer , showing when Option 1 or Option 2 is better . For simplicity , it uses only two dimensions ( mission difficulty and timeliness required ). Option 2 , not Option 1 , is better unless the standard case is assuredly correct . It is possible to show some audiences results with six to 10 independent variables ( Davis , 2014 ) or to project multi-dimensional uncertainty analysis onto two dimensions ( Davis , Bankes , & Egner , 2007 ; Lempert et al ., 2006 ).
Figure 4 conveys a similar story for a more complex case that highlights model uncertainty . It compares simulated outcomes with four different strategies ( called plans in the figure ) given uncertainty and disagreement about which of two models , M1 or M2 , is correct . Reflecting myriad simulations , Figure 4 shows the expected outcome in terms of regret for each alternative strategy as a function of the odds that the first model is correct . “ Regret ” measures how much better one might have done with a different decision . In Figure 4 , strategy B performs best ( lowest regret ) when M2 is correct ( bottom left ) and strategy D has the least expected regret when M1 is most likely correct ( right side ). Strategy A is dominated at every point by other strategies . Strategy C is interesting ; although it is never the best , it places a close second everywhere , with low expected regret . That is , it hedges well . Adopting Strategy C could avoid a costly blunder ( the result of guessing wrong about which model to assume ).
As mentioned earlier , decision-makers have a hard time dealing with contingent predictions for reasons that lie deep in neuropsychology , but we know from personal experience that analysts and senior leaders can learn to think in terms of region charts and tradeoff charts , such as in Figures 3 and 4 . They can then internalize the contingent nature of outcomes . Thus , analyses need to be simplistic , and displays must be intuitive .
5 . Existence Proofs
This paper stems from concern
that analysis does not routinely address model uncertainty . Nonetheless , enough past examples exist to demonstrate feasibility and value . We mention a few briefly .
1 . Planning Military Capabilities . In recent decades , the U . S . Department of Defense ( DoD ) has moved away from planning based on single scenarios toward an approach that seeks to assure that capabilities are adequate to deal with a broad range of possible conflicts and crises ( Rumsfeld , 2001 ) and an understanding of a broad range of ways in
192