Oh, What a Year…Again

Dec 28, 2021

Responding to Assessment and Accountability Challenges During an Ongoing Pandemic

I titled my 2020 year-end CenterLine post, “Oh, What a Year!” with hopes that we were turning the pandemic corner. There’s no question the widespread availability of vaccines beginning in early spring 2021 gave us all a sense of hope only to have it dashed and renewed multiple times throughout this year following COVID’s ebb and flow. The location of COVID hot spots, along with differing political responses, led to considerable variation across states and school districts in patterns of remote versus in-person instruction as we moved through the 2020-2021 school year. In many ways, the term “variability” best describes schooling and testing conditions this year. I’m proud of how my Center colleagues responded to support our state and district partners through this ambiguity.

State Testing in 2021

The year started with arguments over federal testing mandates because of concerns about all of the uncertainties associated with the potential of low participation rates and whether students would even be in school to test. In spite of these disagreements, the Center professionals started early to help states and assessment companies prepare for analyzing and interpreting spring 2021 test scores. Leslie Keng and colleagues published an oft-cited paper with advice for how to prepare for and analyze 2021 state tests. Similarly, Chris Domaleski and I offered a range of alternatives for state testing in this paper. Ultimately, only a limited number of states were able to depart from standard testing practices, such as not testing in every grade.

However, once it became clear that many states would test a good proportion of their students, Center professionals led the way on how to best analyze these data to support defensible interpretations and uses. Damian Betebenner, Nathan Dadey, Leslie Keng, and several other colleagues conceptualized and carried out analyses in more than a dozen states to have a clearer picture of the effects on student learning from the disruptions compared to normal schooling, well summarized in this post by Damian and colleagues. Not resting on our laurels, we’ve already started preparing for 2022 test analyses.

Accountability Support and Reform

As most know, statewide assessment and accountability were suspended during the 2019-2020 school year. While states made valiant attempts to administer state assessments this past spring, essentially all states took advantage of federal flexibility regarding accountability. States were generally not permitted to completely waive accountability determinations and school identifications so they benefited from guidance issued by Juan D’Brot, Erika Landl, Chris Domaleski, and Chris Brandt in partnership with CCSSO. 

Chris Domaleski continued to provide accountability leadership throughout the year with several posts and papers. As Chris noted, restarting accountability systems will be incremental and not immediate. Nevertheless, we are thrilled to see several of our clients and partners—notably Chicago Public Schools, Utah State Board of Education, Nebraska Department of Education, and now Boston Public Schools—engaging in significant accountability reform initiatives. I have written many times about how current accountability systems thwart serious efforts to innovate learning and assessment systems. Therefore, we are hopeful that these developing systems can better support school improvement efforts and serve as exemplars for a reauthorized federal education law. 

Assessment Literacy  

The shift to remote and hybrid schooling up-ended many instructional routines. Enacting formative assessment is challenging in normal conditions but trying to do so over Zoom pushes the limits of even our best teachers. Carla Evans offered helpful suggestions for using formative assessment practices—for both in-person and remote instruction—to help support student learning during the pandemic. 

States and school districts have increased the urgency with which they are trying to enhance the assessment literacy of educators and leaders for both classroom and balanced assessment systems. Jeri Thompson and Carla Evans have been working with numerous states, particularly South Dakota, Louisiana, New Mexico, and Wyoming, as well as with partner organizations, such as the California Collaborative for Educational Excellence, creating structures and building expertise to enhance teachers’ capacity. Additionally, Erika Landl has been co-facilitating (with Caroline Wylie and Scott Norton) CCSSO’s new Balanced Assessment System state collaborative to help leaders support balanced assessment systems in their states.  

Conferences and Internal Work

After missing a year, we hosted The Brian Gong Colloquium, which is our annual internal-facing meeting where we interact with and learn from experts in a particular area. This year we focused, justifiably, on school improvement and accelerated learning research and practice. While the experience would have been better in person, we were fortunate to have about a dozen top experts join us remotely to help us think about how the assessment and accountability systems we design can support school improvement initiatives.

We held our Reidy Interactive Lecture Series (RILS) remotely again this year. Jeri Thompson and Damian Betebenner organized a terrific conference focused on design innovation in educational assessment. We had a fabulous set of both Center and external experts leading us through the many facets of innovative design. All of the recordings and materials can be found here.

We have been concerned about the rapid proliferation of “through year” or “through course” assessment systems because, while such systems seem intuitively sensible, there is essentially no evidence to support the bold claims accompanying these systems. I had the crazy idea back in May to host a convening of state leaders, assessment company representatives, and researchers so we might better understand proposed designs, claims and assumptions behind these designs, and the evidence that might be necessary to support such claims. Thankfully, Nathan Dadey, Brian Gong, and Will Lorié, all of whom have been working with states considering and/or developing such systems, agreed to lead this convening. We are grateful for the many superb participants who gave generously of their time and expertise over the course of four webinars.

Our Team

There are few things more pleasing than hearing how one of our staff members made a big impact on a particular project, meeting, and/or other professional interaction. The great thing about leading the Center for Assessment is that I hear this regularly. Center professionals took on more projects in response to state and district needs and, in spite of working at a tireless pace, my colleagues still managed to share their important ideas through regular blog posts, papers, and presentations. If you haven’t been keeping up, bookmark CenterLine and subscribe to our monthly newsletter to stay up with our latest posts.

Not only are our staff members amazing professionals, but we also genuinely enjoy being with one another. After more than a year without seeing each other except through Hollywood Squares, we had an in-person summer retreat on the New Hampshire Seacoast and another in-person staff meeting just a few weeks ago. We were lucky to time these get-togethers in the troughs of the crazy COVID cycles. We’re hoping to spend more time together in 2022. In addition to social connections, it allows us to do deeper cooperative intellectual work than is possible digitally.

New Team Members

We’re thrilled to welcome two new professional staff members: Laura Pinsonneault and André Rupp. In addition to their impressive professional resumes, they are wonderful people. I know you’ll enjoy getting to work with them. 

Laura was most recently the Accountability Director for the Wisconsin Department of Public Instruction and had worked at WI DPI for more than 10 years. She had been an active member in CCSSO’s Accountability Systems and Reporting (ASR) state collaborative—the group that Chris Domaleski advises—for many years. Chris noted that Laura was one of those state leaders who other ASR members regularly turned to for leadership and guidance. Laura is a skilled facilitator and with her deep knowledge of accountability systems, she’s already joined a variety of projects with several Center senior staff.

André has already had a distinguished career as a faculty member at University of Maryland, senior researcher at ETS, and most recently as the assessment director at Success Academy. Besides adding considerable technical expertise to our already stellar team, André is one of the deepest thinkers about the applications of educational measurement in our field. He’s already made contributions to several projects, and we are excited about what he will add to the Center and the field over the coming years.

Thank You

Anyone following education knows the physical and emotional toll this pandemic has had on students and teachers. Often unseen, however, this burden was felt by those even a few steps removed from the classroom. I’m in awe of the tireless efforts put forth by state and district assessment and accountability leaders. It’s not enough to just have a “Plan B” anymore. Our colleagues and partners have to think about Plans C, D, and E, all with multiple contingencies. 

Richard Feynman once said being a scientist is to be in love with uncertainty. As a scientist, I usually resonate with this perspective, but I wish for a bit more certainty for all of you in the coming year. Please stay safe and sane and thank you for all you do for our nation’s children.