European Policy Analysis Volume 2, Number 1, Spring 2016 | Page 117

European Policy Analysis

The curve in the left partial
dependence plot starts close to zero.
This means that in the early 1950s,
punctuations were quite likely. The
chance of punctuations started decreasing
then, but it rapidly rose at the end of the
1990s. The congress plot shows us that the
attention of the Congress on a topic lowers
the possibilities of dramatic budget shifts.
This is absolutely in line with PET, because
frequent attention on a topic prevents
policy bubbles. But if attention rises too
much, this can be a cause for punctuations
as well. As we can see, decision trees are
very good tools to deal with such nonlinear
effects.

Conclusion

T

he results have shown that random
forest is a very powerful algorithm
to analyze extreme values. “Random
forests are an effective tool in prediction.
Because of the Law of Large Numbers
they do not overfit. Injecting the right
kind of randomness makes them accurate
classifiers and regressors” (Breiman 2001,
29).

Taking the potential of machine
learning into consideration, political
science should welcome these approaches
where complex data is to be analyzed.
The advantages are the ability to deal
with multiple—even highly correlated—
predictors, the sensitivity to nonlinear
effects—including contradictory effects in
different regions of the predictor space—
and the possibility of analyzing unbalanced
data, where one class strongly outnumbers
the others. But these advantages come
with a price that is not limited to the extra
effort necessary to learn new and complex
methods: “There is no free lunch in statistics:

no one method dominates all others over
all possible data sets” (James et al. 2013,
29). Most importantly, there is a trade-off
between prediction accuracy and model
interpretability. While decision trees are
quite easy to explain,17 random forests are
much harder to interpret. Often, political
science is more interested in inference
than in accuracy, which sets a natural limit
to the scenarios this approach might be
implemented successfully. In addition, if
applied as “black-box-algorithm” without
a deeper understanding of the inner
mechanism, random forests might lead to
misinterpretations and false discoveries.
But this should be seen as strong argument
for political scientists to engage in these
“new” methods. In the big data world,
the machine learning algorithm will
become more and more popular. Hastily
conclusions from models that are accurate
but lack a deeper understanding of the
political context can only be criticized by
scientists who are familiar with the subject
as well as with the method.

Notes
There are scenarios in which the distinction
between supervised and unsupervised
might not be as clear as indicated here, for
example, when there is a response variable
but only for some cases. James et al. use the
term “semi-supervised learning” for those
kinds of problems (James et al. 2013).
2
A good example is the “bag of words”
approach in text mining. Here, every word
that is present in any of the documents
of the corpus is taken as one predictor.
Therefore, the number of predictors
will often outnumber the number of
documents.
1

117