16 RESEARCH AT IOB
( here: the months before the FCL arrangements were agreed) match as closely as possible those of the treated unit. This matching is performed by an optimization algorithm( available in Stata, R or MATLAB).
In short, the synthetic control method performs a data-driven comparative case study, where instead of one comparator unit, a‘ synthetic version’ or combination of comparator units is used. This synthetic control typically provides a better counterfactual than every single control unit in itself.
E2C: Under which conditions might it be optimal to use the SCM?( Why did you decide to use it in your research project?)
A well-known problem in research at a highly aggregated level, be it in macroeconomics or development studies, is the difficulty to establish causality, from a particular intervention to outcome variables. For example, one cannot run randomized control trials( RCTs) or other types of experiments on countries. So-called‘ quasiexperimental’ research addresses this problem by constructing treatment and control groups in datasets where such groups did not exist originally( because the intervention was not assigned at random). As one of various possible quasi-experimental techniques, the SCM method is particularly suited for research that focuses on a single or limited number of interventions that can be expected to have sizeable effects on a single or limited number of treated units( countries, regions, villages, etc.), and when only a limited number of potential control units are available.
E2C: How does the synthetic control method differ from other well-known quantitative impact evaluation methods such as Propensity Score Matching( PSM) or Difference-in-Difference estimation( DID)? What are the comparative advantages?
Unlike PSM and DID, the SCM does not need large samples of treated and untreated units. Another important difference with PSM and DID is that the SCM can take into account unobserved heterogeneity that varies over time. It does so by incorporating pre-intervention outcomes in the matching procedure. Moreover, the SCM makes explicit the relative contributions of different control units to the counterfactual, allowing the researcher( and others) to evaluate the relevance of the comparisons undertaken. For example, I find that a weighted combination of Chilean, Brazilian, Hungarian, South African and Salvadoran bond spreads provides a good counterfactual of Mexican spreads. This‘ synthetic Mexico’ matches the pre-FCL characteristics of the actual Mexico much better than any emerging market country separately. Finally, because weights are restricted to be positive and smaller than one, extrapolation outside the support of the data is avoided. Typical regression analysis does not impose such restrictions on the weights it( implicitly) assigns and may therefore suffer from extrapolation biases.
E2C: Have you also encountered any drawbacks or limitations when you applied the SCM in your current research project? Like any research method, one supposes it also has some drawbacks. Can you elaborate on this?
A classic drawback to the SCM is that unlike regression analysis, it cannot rely on standard( large sample) inference by means of significance tests or confidence intervals around the estimated effects. One way to circumvent this limitation is by using‘ placebos’, i. e., applying the SCM to untreated units as if they were the treated unit.
Another drawback- which is specific to my research- lies in the“ untreated units are entirely unaffected by the intervention” assumption of the SCM. Some prior studies on the FCL, however, claim that a select group of non- FCL emerging market countries also benefited indirectly from the creation of this new IMF instrument( since investors considered them to be eligible too). If this is indeed the case, then the beneficial effects of FCL that I find would be underestimations of the true effects.
Exchange to change January 2017