16 RESEARCH AT IOB
( here : the months before the FCL arrangements were agreed ) match as closely as possible those of the treated unit . This matching is performed by an optimization algorithm ( available in Stata , R or MATLAB ).
In short , the synthetic control method performs a data-driven comparative case study , where instead of one comparator unit , a ‘ synthetic version ’ or combination of comparator units is used . This synthetic control typically provides a better counterfactual than every single control unit in itself .
E2C : Under which conditions might it be optimal to use the SCM ? ( Why did you decide to use it in your research project ?)
A well-known problem in research at a highly aggregated level , be it in macroeconomics or development studies , is the difficulty to establish causality , from a particular intervention to outcome variables . For example , one cannot run randomized control trials ( RCTs ) or other types of experiments on countries . So-called ‘ quasiexperimental ’ research addresses this problem by constructing treatment and control groups in datasets where such groups did not exist originally ( because the intervention was not assigned at random ). As one of various possible quasi-experimental techniques , the SCM method is particularly suited for research that focuses on a single or limited number of interventions that can be expected to have sizeable effects on a single or limited number of treated units ( countries , regions , villages , etc .), and when only a limited number of potential control units are available .
E2C : How does the synthetic control method differ from other well-known quantitative impact evaluation methods such as Propensity Score Matching ( PSM ) or Difference-in -Difference estimation ( DID )? What are the comparative advantages ?
Unlike PSM and DID , the SCM does not need large samples of treated and untreated units . Another important difference with PSM and DID is that the SCM can take into account unobserved heterogeneity that varies over time . It does so by incorporating pre-intervention outcomes in the matching procedure . Moreover , the SCM makes explicit the relative contributions of different control units to the counterfactual , allowing the researcher ( and others ) to evaluate the relevance of the comparisons undertaken . For example , I find that a weighted combination of Chilean , Brazilian , Hungarian , South African and Salvadoran bond spreads provides a good counterfactual of Mexican spreads . This ‘ synthetic Mexico ’ matches the pre-FCL characteristics of the actual Mexico much better than any emerging market country separately . Finally , because weights are restricted to be positive and smaller than one , extrapolation outside the support of the data is avoided . Typical regression analysis does not impose such restrictions on the weights it ( implicitly ) assigns and may therefore suffer from extrapolation biases .
E2C : Have you also encountered any drawbacks or limitations when you applied the SCM in your current research project ? Like any research method , one supposes it also has some drawbacks . Can you elaborate on this ?
A classic drawback to the SCM is that unlike regression analysis , it cannot rely on standard ( large sample ) inference by means of significance tests or confidence intervals around the estimated effects . One way to circumvent this limitation is by using ‘ placebos ’, i . e ., applying the SCM to untreated units as if they were the treated unit .
Another drawback - which is specific to my research - lies in the “ untreated units are entirely unaffected by the intervention ” assumption of the SCM . Some prior studies on the FCL , however , claim that a select group of non- FCL emerging market countries also benefited indirectly from the creation of this new IMF instrument ( since investors considered them to be eligible too ). If this is indeed the case , then the beneficial effects of FCL that I find would be underestimations of the true effects .
Exchange to change January 2017