Meaningful Measures of Student learning - Page 9

To ensure Tennessee’s assessment is adequately aligned to standards, both in terms of cognitive demand and content coverage, it would be ideal for evidence from multiple research-based alignment methods to be provided. If evidence from multiple alignment methods is unavailable, it is imperative for evidence from at least one rigorous alignment study to be provided. There is no single best approach to measuring alignment and different alignment methods can produce different results. For this reason, it is important to use multiple alignment methods to understand how well aligned the selected assessment is to Tennessee’s State Standards. Table 2 describes three different methods that can be applied to measure how well an assessment aligns to academic standards. Methods of Measuring Alignment Webb The Webb process uses four criteria to judge the alignment between standards and assessments: Depth of knowledge consistency indicates whether assessment items are at the same levels of cognitive demand as is present in the standards. Range of knowledge measures whether standards and assessments cover a similar range of concepts. Achieve Categorical concurrence provides a general indication of whether standards and assessment cover the same content. Balance of representation measures how often one standard is assessed as compared to other standards.43 The Achieve alignment method reports on five criteria:44 Performance centrality measures the degree to which the cognitive demand of a test item matches the cognitive demand required on the standards. Challenge evaluates how well items require students to master challenging subject matter. Balance measures how equally the standards are covered on the assessment. Surveys of Enacted Curriculum (SEC) Content centrality looks at the degree to which the content of test items and the content of the standards align. Range measures how well assessments cover the range of standards.45 The SEC process uses a “topic by cognitive demand” matrix. Reviewers categorize standards and assessments onto the matrix based on the topics covered and levels of cognitive demand in the standards and assessment. The degree of alignment is determined by looking at the degree to which the different matrices overlap. This technique can also be used to examine the content and alignment of curriculum materials or teachers’ instruction.46 Table 2 Each of these alignment methods has strengths and weaknesses. For example, the Webb approach is seen as providing the most detailed quantitative indication of alignment, the Achieve approach is seen as providing the most detailed qualitative indication of alignment, and the SEC approach is the only approach of the three that also considers alignment of instruction and curriculum with standards and assessments.47 In order to gain a more accurate measure of alignment between Tennessee’s standards and assessments, a method could be developed that combines the strengths of these different approaches and that is specifically designed to measure an assessment’s alignment to Tennessee’s current standards. Assessments ensure technical quality. The assessment should be valid, reliable, and fair. In terms of validity, at the most basic level, the assessment should be a good representation of the knowledge and skills it intends to measure. A high-quality alignment study like those in the previous section should ensure adequate content validity.48 Beyond content validity, since standardized assessments serve a variety 9