Comparability is at the core of educational assessment and accountability. When we want to measure changes in a student’s performance from one year to the next, we require evidence that the two assessment scores justifiably can be compared. Similarly, if policymakers want to evaluate the extent to which certain policies are having the intended effects, they must be able to compare apples to apples. Key approaches for ensuring and evaluating comparability of educational assessments; e.g., state test results over time, falls under the umbrella of score linking. The Center for Assessment has nearly two decades of experience working with states and other educational agencies to design and implement effective assessment policies and processes that help promote high-quality linking practices.
The shift to the Common Core State Standards and consortium-based assessments (e.g., PARCC and Smarter Balanced) has increased the focus on comparability and, importantly, on the threats to comparability including, but not limited to: different modes of test administration (computer vs. paper), different types of devices (e.g., tablets, desktop computers), and different administration vendors. Center staff have recently led and/or participated in psychometric and research work for the PARCC consortium and states using PARCC content (i.e., Louisiana, Massachusetts) such as conducting analyses and advising on issues related to comparability of test scores across test forms with varying proportions of PARCC content, different administration modes, and other conditions that may impact the comparability claims that can be made about the test scores.
Having deep expertise and familiarity with approaches from other jurisdictions allows Center professionals to respond to novel or otherwise challenging contexts. This is the case for evaluating comparability claims within the assessment consortium contexts, and for designing comparability evaluations for innovative assessment systems.
Some recent comparability publications include:
- Charles DePascale, Nathan Dadey, & Susan Lyons (2016). Score Comparability across Computerized Assessment Delivery Devices
- Scott Marion & Susan Lyons (2016). Comparability options for state applying for the Innovative Assessment and Accountability Demonstration Authority: Comments submitted to the United States Department of Education regarding proposed ESSA regulations
- Brian Gong & Charles DePascale (2013). Different But the Same: Assessment "comparability" in the era of the Common Core State Standards