The Center at NCSA 2018

Jul 26, 2018


State assessment teams, assessment industry staff, and other assessment specialists gather each June at the CCSSO National Conference on Student Assessment.  Historically, the annual conference provides an opportunity for the Center team and our partners to share innovative solutions and our latest thinking on the most pressing assessment and accountability issues of the day. This year, seven Center team members participated in eleven sessions over the three-day conference: Chris Domaleski, Carla Evans, Brian Gong, Leslie Keng, Erika Landl, Scott Marion, and Joseph Martineau

The topics they addressed reflect the key issues that have been the focus of the Center’s work over the course of the last year:

  • Designing and aligning assessments of the Next Generation Science Standards
  • Moving beyond traditional stand-alone state assessments to create innovative and comprehensive assessment systems.
  • Establishing and validating performance standards for complex assessment and accountability systems
  • Promoting the effective use of data and information from state assessment and accountability programs.

In the paragraphs below we summarize these issues that will continue to be a focus of our work in the year ahead.  This is followed by a list of sessions at NCSA related to these issues.  Each listed session is linked to the online program listing, where additional details and resources can be accessed. 

Assessments for the Next Generation Science Standards

The NGSS are our most complex set of K-12 content standards.  The combination of their 3-dimensional nature and their organization around a set of Performance Expectations challenge our current best practices in assessment design and development.  Traditional conceptions of alignment and methods for determining the alignment of assessments and standards are not suited for the complex domain representations found in the NGSS.  Performance Expectations pose a particularly tricky dilemma, simultaneously representing only a portion of the full set of the NGSS, but much more than can be assessed on an on-demand state assessment.  Only through principled design of comprehensive assessment systems will we fully realize the implementation of the NGSS at the state, district, and classroom levels.

NCSA Sessions

Innovative and Comprehensive Assessment Systems

Innovative assessment and accountability systems are important because they have the potential to re-align state assessment systems in such a way that there is coherence between the underlying theory of learning, goals, and purposes for the assessment system, and the design of the assessment system. This allows for positive feedback loops to occur through the curriculum, instruction, and assessment cycle, and for efficiency in the number of assessments required to inform stakeholders about students’ progress towards proficiency. The Center’s current work focuses on specifying the purpose of the system as a whole and the function of each of the components and on understanding the practical considerations related to the design and implementation of innovative and comprehensive assessment systems.

NCSA Sessions 

Establishing and validating performance standards for complex assessment and accountability systems

Traditional assessment standard setting procedures have come to rely on procedural fidelity as the primary evidence of validity. A systems approach to assessment and accountability design, however, requires a systems approach to standard setting. Assessment and accountability systems can be complex often incorporate multiple indicators for students and schools. Often these indicators are combined and results to a single culminating achievement level or performance rating.  Given the importance of these ratings, there should be compelling evidence that they have a high degree of validity for the intended interpretation and uses. A substantial part of that validity argument is the design and implementation of a sound process for establishing standards that credibly reflects the state’s vision for their assessment and accountability system.  

For example, a central priority for most SEAs is to promote student readiness for college and careers, which informs the design of K-12 assessment and accountability systems. To validate these assessment and accountability systems, SEAs must reach beyond K-12 data to evaluate postsecondary student outcomes. Higher Education also relies on K-12 data, such as when educator preparation programs seek to evaluate the efficacy of their teacher preparation programs by analyzing K-12 assessment scores for their program completers. 

Equally important, is the need for standards validation after implementation and again after modifications to the systems. While standards validation is common practice, clear criteria do not exist to inform when and how these reviews should be done or to evaluate the effectiveness of the process in providing for more “appropriate” standards. 

NCSA Sessions:

Promoting the Effective Use of Information from Assessment and Accountability Systems

Finally, arguably the most important step in the process of designing and implementing assessment and accountability systems is to ensure that they provide information that can be used to improve instructional practices and student achievement.  For example, most states, under ESSA, have made a primary goal of reducing achievement gaps. How can current assessment and accountability systems be designed to provide more useful information for guiding efforts to reduce achievement gaps, rather than merely describing and documenting them? A framework applicable by every state that provides definitions and guidance for incorporating excellence and equity into a theory of action focused on practical results is one way that helps states to link state reports with district and school actions on key factors of equitable opportunity and access. 

NCSA Session: