Global Security and Intelligence Studies Volume 5, Number 1, Spring / Summer 2020 | Page 103

The Challenge of Evaluating and Testing Critical Thinking in Potential Intelligence Analysts
Figure 3: Production framework of solution fluency (Crockett 2012).
by each individual professor. Yet, critical
thinking is not entirely dependent on
the skill base developed in college, but
ideally should be developed throughout
secondary education. Interestingly,
the College Board revamped the SATs,
which are taken when students are between
sixteen and seventeen years of
age, to better assess a high school student’s
critical thinking (Willingham
2008). In secondary education, due to
the reliance on standardized testing
(e.g., SATs and the required Standards
of Learning (SOL) tests), which hinges
on multiple-choice questions, schools
do not develop lesson plans around
building critical thinking skillsets, but
on being able to recall facts, which is a
low-level process.
Additionally, it has been proposed
that the process of thinking is
intertwined with domain knowledge.
Anything experienced is automatically
interpreted from what a student (or
employee) already knows about similar
subjects. Familiarity with a problem’s
deep structure and the knowledge that
one should look for a deep structure
is inherent in critical thinkers. When
a student or employee is very familiar
with a problem’s deep structure, knowledge
about how to solve it transfers
well. That familiarity can come from
long-term, repeated experience with
one problem, or from various manifestations
of one type of problem (i.e.,
many problems that have different surface
structures, but the same deep structure).
After repeated exposure to either
or both, the student or analyst simply
perceives the deep structure as part
of the problem description. However,
it takes a good deal of practice with a
problem type before a person knows it
well enough to immediately recognize
its deep structure (Willingham 2007).
Many classes at the university
level adopt meta-cognitive critical
thinking paradigms to coursework
where a problem’s deeper structure is
explored. For example, critical thinking
can be embedded into scaffolded as-
89