School Improvement is Hard! How Can Educational Assessment and Accountability Systems Help?

Building a Better Theory of Action to Enhance the Utility of Assessment and Accountability Systems

How do schools improve? More importantly, how can school improvement be sustained over time and at scale? 

Here at the Center for Assessment, we do not generally deal directly with these questions. Being a relatively small organization of leading assessment and accountability experts, we understand the importance of “staying in our lane” to avoid mission creep. However, heeding the advice Juan D’Brot offered in a recent CenterLine post, if we’re in the business of designing high-quality assessment and accountability systems putatively to improve school quality and, ultimately, student learning, it will help to have a deep understanding of what school improvement really requires. 

The Center’s Early Efforts Toward School Improvement

Shortly after its founding, the Center for Assessment conducted a study to figure out the magnitude of test score changes that might be possible in schools undergoing improvement. Since the Center was responsible for helping states set accountability goals, we wanted to understand what was possible. The original study did not seek to understand the mechanisms of school improvement; we simply wanted to know what was possible in order to set, in the words of Bob Linn, ambitious, but reasonable goals.

During the past 20 years, since that first study, we have been creating accountability systems designed to support school improvement activities within a set of narrowly-defined, test-based constraints. The federal government and states have taken some steps to expand accountability systems beyond an exclusive focus on student performance on state assessments in English language arts and mathematics. We now think it is time for us to better understand the processes and mechanisms by which schools improve if we’re going to design assessment and accountability systems that can contribute positively to school improvement.

To better understand the nature of information necessary for helping improve teaching and learning in schools, we have identified three major areas of inquiry for 2020. 

Learning about School Improvement

Virtually everything we do at the Center is based on a theory of action, and the starting point for developing a theory of action is to identify: 

• the intended goals and outcomes (i.e., the purpose of the system), 
• the stakeholders charged with achieving the goals, 
• the processes and mechanisms necessary to achieve the goals,
• intermediate indicators to help evaluate progress toward the goals, and
• the data and information they need to help them meet the goals.

Therefore, we will start with a deep dive into school improvement from both research and practical perspectives. We will explore such topics as:

• What does it mean to improve schools? 
• What do the research and practice say about what it takes to implement sustainable school improvement strategies?
• What are the most common models of successful school improvement?
• How does context influence improvement and turnaround? 
• Can we point to cases where this work has been accomplished at scale and been sustained over time?

The Role of Data in School Improvement

All of the school improvement models we’ve studied discuss the role of data and information at various places in the school improvement cycle, some more explicitly than others. We at the Center for Assessment are involved in helping states and school districts generate and use a variety of data for a variety of purposes.  

We know many school improvement models (e.g., Networked Improvement Communities) rely on data at multiple points in the improvement cycle (e.g., identifying needs, evaluating processes and outcomes) and we intend to explore the different types of data necessary at different points in the cycle. 

Assessment and Accountability System Design

With the foundation of school improvement and data needs established, we return our attention to accountability and assessment system design. 

The Center for Assessment has focused for many years on the design and implementation of balanced assessment systems to better support multiple uses and needs for assessment information. We will consider how the components of high-quality assessment systems at multiple levels (e.g., state, district, and classroom) can contribute to various models of school improvement. 

Accountability systems transform assessment and other data to produce indicators intended to signal factors on which policymakers would like schools to focus. Data are often combined through transformation processes, such as combining the mathematics, English language arts, and science assessment results across all grades into an achievement indicator. Such practices are useful for providing a general picture of student achievement at the school level, but make it hard to identify specific improvement needs.

A few states and school districts have developed reporting systems to provide useful information at various points throughout a clearly-articulated theory of action. We often refer to these as “little a” accountability. For example, low high school mathematics scores might be due to a lack of students enrolled in advanced math courses, which might, in turn, be due to both tracking and weak performance in prerequisite math coursework. Understanding the originally-identified “problem” requires more fine-grained information throughout the system to be able to target interventions where they are most needed. 

Closing Thoughts

We have seen short-lived and sporadic changes in student achievement since the passage of No Child Left Behind (NCLB). 

We recognize that many of the factors associated with this lack of improvement are related to structural factors beyond the control of school and district personnel. We also recognize that to some degree, the lack of improvement is related to a failure to get useful information into the hands of educators who are working on a daily basis to improve schools and the policymakers who are entrusted with the responsibility of supporting their efforts.

Without ignoring key structural issues, our goal is to improve our capacity to support meaningful school improvement in our roles as expert consultants in assessment and accountability.