Global Security and Intelligence Studies Volume 5, Number 1, Spring / Summer 2020 | Page 111

The Challenge of Evaluating and Testing Critical Thinking in Potential Intelligence Analysts
TEST A Time Frame: 30-60
minutes. Cost: $28.00 per
person & $37 per profile
development (20 tests =
$1,300).
Measures ability to draw
conclusions. Reflective
component allows
participants to provide
assessment and feedback.
Purports to measure
confirmation biases and
emotional thinking.
Test questions generic to
critical thinking; no defense
scenarios.
Provides a general overview
of the candidate: moderate,
strong, or weak critical
thinking skills.
Provides subscale
interpretation for recognizing
assumptions, evaluation of
arguments, and drawing
conclusions in a nonnumerical,
detail-rich report.
Medium length test with
non-military scenarios.
CRITICAL THINKING TEST ASSESSMENTS
TEST B Cost: E-testing
System Orientation - $190.00;
20 Defense Skills at $75.00
each = $1,690.00
Two-part test measures
critical thinking and
personality traits
corresponding to
participants’ critical thinking
ability.
The two tests give a solid
overview of skills and ability.
Test questions are specific
with many defense/military
scenarios that use critiquing
and justifying decisions in
scenarios.
Provides overall numerical
score for critical thinking and
a descriptive interpretation.
Provides subscale
interpretation for ambiguous
contexts, precise contexts,
problem analysis, quantitative
contexts, and evaluation of
alternatives with a detail-rich
report.
Test is longer in duration
with in-depth scenarios;
second part evaluates how the
tester approaches different
scenarios.
TEST C Time Frame: 20-
45 minutes. Test System
Hardware: $116.88, Test
System Software: $532.91,
Postage/Packing: $146.10 =
$795.89
Tests critical thinking in
an easy to understand
way. Judgments applied to
everyday scenarios.
Purports to measure critical
thinking skills involved when
confronted with a general
scenario.
Test questions generic to
critical thinking; no scenarios
specific to defense.
Provides an overall numerical
score of critical thinking.
Does not provide subscale
interpretation of verbal
reasoning, argument
analysis, skills in thinking
as hypothesis testing, using
likelihood and uncertainty,
decision-making, and
problem solving.
Short test length with nonmilitary
scenarios.
Figure 7: Critical thinking test assessments (Marangione 2019).
The DIA also administered Test
A to a sample of its employees. The
concerns that the DIA had regarding
the test was how effective the test was
in measuring metacognition—thinking
about thinking—or measuring whether
a potential analyst is capable and aware
of the processes used to plan, monitor,
and assess one’s understanding (Moore
2007). Test A and B, as determined by
97