How RILS Helps Define a Path Forward for Educational Assessment and Accountability

A Vision for More Effective Practices in the Future

This post is one of several recapping the Reidy Interactive Lecture Series (RILS), which the Center for Assessment held on September 27-28, 2018. This event marked the Center’s 20th anniversary, and we took advantage of the milestone to reflect on the past and look ahead to the future. In particular, we wanted to take an honest look at what we’ve learned over the years and leverage these lessons to create a vision for more effective assessment and accountability practices in the future. The following are insights from RILS regarding how the event helps identify a path forward for the field.

Assessment and Accountability Today: Have We Lost Our Way?  

It may be useful to begin by examining the current state of assessment and accountability. Frankly, many worry that the field has been lost in the wilderness in recent years.   

It’s no secret that assessment and accountability are charged terms to many. But why?

One prominent reason is the growth and ultimately the dominance of test-based accountability systems that are heavily–even exclusively–mandated by state and federal legislatures. The tests central to these systems are often seen as partial measures of the skills that matter the most. What’s more, they are perceived as burdensome to students and for educators to administer, consuming time and resources that could be better spent supporting teaching and learning. Finally, assessment and accountability practices seem to be constantly shifting. Many schools are fatigued and frustrated by measures and performance targets that seem to change annually. Is it any wonder that many educators have lost their confidence in assessment and accountability?

Where Do We Go From Here?

If our current situation is far from ideal, how do we move forward? I believe the discussions at RILS suggest there are three prominent behavior markers that indicate we are heading in the right direction:   

• A focus on assessment systems
• An effort to restore balance in accountability
• An emphasis on evaluation and refinement

A focus on assessment systems: Currently, assessment and accountability are dominated by end-of-year summative assessments, primarily created to fulfill the requirements of the Every Student Succeeds Act (ESSA). These summative assessments are not well suited and are not designed to provide detailed information about student performance to inform instruction.

Summative assessments are chiefly designed to provide high-level performance classifications with respect to a wide range of state standards. To the extent we ask a single summative assessment to ‘do more,’ such as measure more standards or assess more deeply, we risk diminishing returns or producing counterproductive outcomes. Assessments designed to do multiple things well rarely do any of those things well. Besides, there is little tolerance for summative assessments that are lengthier, more expensive, and that take longer to provide results.

A better alternative is to focus on a system of assessments that is balanced, coherent, and more useful to inform instruction. Center Executive Director Scott Marion and his colleagues discuss the principles and characteristics of such systems in their aptly-titled paper A Tricky Balance. These systems can be designed to focus on local curriculums, producing timely and useful information to inform instruction.  In aggregate, these systems are more likely to be comprehensive–better measuring the full breadth and depth of what matters for multiple users. Moreover, thoughtfully-designed systems, built on a logical sequence of learning (i.e. learning progressions) are more efficient and useful to educators and students.  

Making progress starts with capacity-building in districts and schools, where the direct influence on classroom assessment resides. Initiatives such as New Hampshire’s Performance Assessment of Competency Education (PACE) offer promising examples of the benefit of empowering and equipping educators to develop high-quality performance assessments that provide more meaningful and timely feedback to inform teaching and learning.

Restore balance in accountability: We cannot leverage the promise of improved assessment systems without similar innovations in accountability. Currently, federal and state accountability systems have an outsized influence on practice. We propose focusing on approaches to measure what matters in districts and schools, and using this information to evaluate and support ongoing improvement at the local level.  

As my colleagues and I explain in our paper on Coherent and Balanced Accountability Systems, local initiatives can complement existing systems to create more comprehensive and effective accountability practices. Such systems reflect principled design linked to clear goals, provide support structures to meet performance expectations, include district-specific measures, and are appropriately differentiated at each level.

For example, local systems can include indicators that reflect academic information tied to the district’s curriculum, reflecting the unique mission and characteristics of the schools and districts. Balanced accountability should not involve competing systems; rather, the relationships among federal, state, and local systems are considered together to create a coherent approach to support improvement.   

Discussion at RILS highlighted some promising approaches to local accountability, such as the Student-Centered Accountability Program in Colorado (S-CAP) and the Massachusetts Consortium for Innovative Education Assessment (MCIEA).  

Evaluate and refine: As we expand our expectations for assessment and accountability to include new terms (or new uses for old terms) like coherent, comprehensive, and comparable, it is critical to developing criteria and methods to evaluate and refine our progress.   

In the best case, evaluations should:

• Be tailored to the goals and purposes of the system
• Examine the conditions and assumptions necessary to achieve target outcomes
• Identify and the near and long-term outcomes that should be observed. (By so doing, we can refine our assumptions, adapt the designs, and improve implementation to better achieve the intended system outcomes.)   

Encouragingly, innovations in evaluations are being developed to address these new system conceptualizations, such as Senior Associate Juan D’Brot’s approach to accountability evaluation and Senior Associate Leslie Keng’s work on validation in the midst of change. These and other works acknowledge the importance of collecting evidence to study the full range of claims for assessment and accountability systems in order to continuously improve practice.  

Final Thoughts

I acknowledge the ideas discussed in this paper are not simple, quick fixes. Indeed, the current concerns with assessment and accountability are substantial and, I believe, require more than ‘tinkering around the edges.’   

There is much more to be done, but I see encouraging signs that state-district partnerships to build capacity to improve assessment and accountability can produce impressive results. I also recognize that progress relies on collaboration among professionals in curriculum, instruction, and assessment to design solutions that are better than any individual in one of these areas can produce on their own.

Finding a path forward will require a distinct change in direction and a steady effort.  However, I believe there is cause for optimism–in no small part due to the insights and promising practices discussed at RILS this year.