Global Security and Intelligence Studies Volume 4, Number 1, Spring/Summer 2019 | Page 24

Forging Consensus? Approaches to Assessment in Intelligence Studies Programs
With that in mind, it is useful to compare the range of learning outcomes
that are identified by the programs in this study. These program objectives should
identify the priorities of each academic program. As Coulthart and Crosston note,
these objectives identify outcomes that transcend a single course. That is, due to
their importance, the program seeks to repeat and reinforce an essential learning
or skill objective for as much as 144 weeks instead of a single 12-week period
(Coulthart and Crosston 2015, 65). If there is wide variation among the sample
programs in their student learning objectives, a wider variety of assessment structures
among the sample would be expected.
In their study of course descriptions in U.S. intelligence studies programs,
the Coulthart and Crosston study identifies three key areas of focus for program
objectives related to student learning. First, they identify procedural areas of
knowledge such as communication and analytic tradecraft. These areas focus on
the higher level learning objectives in Bloom’s Taxonomy, particularly application.
The second area that is identified is core knowledge, which focuses on historical
issues of the field, as well as current actors, processes, and legal issues that explain
contemporary intelligence activities. The last area that Coulthart and Crosston
note is domain knowledge which covers the subject matter context in which
intelligence activities occur. This would include the study of international relations,
domestic criminal justice issues, or the private sector business environment
(Coulthart and Crosston 2015, 57).
This study has utilized this framework with one modification. Ethical and
legal issues are identified as an area of core knowledge in the Coulthart and Crosston
study. Certainly, knowledge of legal frameworks falls squarely within this
realm. However, ethical dilemmas involve a form of reasoning that is normative
in character and beyond the scope of subject matter expertise. Several programs
strive for students to “understand” or “apply” ethical reasoning, integrity or moral
standards. For instance, the programs at Angelo State specify both “ethical
awareness” and “moral reasoning” in their SLOs (Tony Mullis, E-Mail Message to
Author, January 25, 2018). So, this study reviewed the sample programs for this
fourth knowledge area, as well.
As can be seen in Table 1, what is found among the sample is a high degree
of uniformity among the sample programs. Regardless of level or mode of delivery,
there was uniformity across the areas of procedural, core, and domain knowledge.
There is some variation with regard to the area of ethical reasoning. Two of the six
institutions identified this as a student learning objective. But with that exception,
there is a high degree of similarity between program objectives.
This similarity in program outcomes is very useful in our exploration of
assessment structures in intelligence studies programs. Consistent with the past
scholarship (e.g., Spracher 2009), the programs in this study are largely purport
to achieve the same objectives. That is, the expected benefits of being in one pro-
13