Confronting Model Uncertainty in Policy Analysis for Complex Systems : What Policymakers Should Demand
This will mean analysis to find strategies that are flexible , adaptive , and robust ( often called robust for short )— i . e ., strategies that are expected to do well across the range of assumptions about inputs , how the world works , and how alternative policies would affect outcomes over time . The need to do so should become a basic ethic for analysts of choice under uncertainty ( Davis , 2014 ).
Success will require basic cultural changes in the analytic and policymaker communities using analysis . Table 3 characterizes the needed culture changes . The first column shows common current-day questions asked by policymakers and by people developing terms of reference for studies . The second column indicates the better types of question that would represent a shift in analytic culture .
Table 3 . Culture Changes Needed for Analysis Under Uncertainty
Current Type of Question Asked by Policymakers
What ' s your best prediction ? What should we prepare for ?
Which option is best ?
Whose options are you going to look at ?
What data are you using ?
Do you have a steering group with reps from the relevant offices ?
Are you using a validated model and data ?
Better Questions for Policy Under Uncertainty
What are the things I can do now that will best position us to deal with whatever arises down the pike ?
How do I make a robustly good decision ? What is necessary to have a strategy that is as flexible , adaptive , and robust as possible given budgetary and other considerations ?
Where will your options come from ? Are you talking to everyone ? Are you going beyond the options that the bureaucracy provides ? Will there be new ideas ?
What is your campaign plan for analysis ? Will you be addressing the fundamental uncertainties and disagreements , as well as the uncertainties of so-called data ( are the right variables being measured is the data credible )?
Do you plan to have a “ Red team ” to ask the hard questions ? Do you plan to have an Advisory Group to assure that you ' re covering all the bases — a group of not just the “ official ” stakeholders , but of everyone who should be heard ?
Will implementation be part of the campaign of analysis or will that be an afterthought ?
How is your campaign plan for analysis going to assure that your modeling and analysis are valid for the purposes intended , whether to " describe and explain ," to " explore to find insights and the ingredients of robust decisions ," or prediction ? What model ( s ) are you going to consider ?
195