2021 Reidy Interactive Lecture Series: Design Innovation in Educational Assessment Systems

September 2021

Held each fall since 1999, the Reidy Interactive Lecture Series (RILS) provides an opportunity for a diverse group of participants to discuss timely and important topics in assessment and accountability. This year, RILS will continue as a no-cost, virtual offering that we hope will sustain the great attendance and participation we saw last year. RILS 2021 will comprise six sessions on four different days. Details of each session and registration are as follows:

Session 1: Introduction to Design InnovationDesign and innovation in the creation of educational assessment systems.

Wednesday, September 15, 2021; 1:00-2:30 p.m.

Guest Presenter: Charles DePascale

Drawing on his own experiences with innovation and educational assessment over the past six decades, Charlie will offer a brief look at the history of innovation in large-scale educational assessment – which inventions became innovations and which became just the latest fad? When was innovation in educational assessment driven by Design Thinking and what difference did that make? What separates recent attempts at innovation in educational assessment from just about every successful innovation?

Webinar Recording

Session 1: Introduction to Design Innovation

Session 2: Understanding the Problems for Design Innovation - Root Cause Analysis.

Monday, September 20, 2021; 1:00-2:30 p.m.

Center Staff: Juan D’Brot and Chris Brandt

Designing a balanced assessment system that meets the various needs of stakeholders requires both an understanding of the context as well as the problem that needs to be solved through the redesign of the assessment system. In order to support a redesign, the problems must be well-specified and focused, which may require breaking apart the larger problems into manageable components, supporting more effective root cause analyses, and helping to specify a theory of action. This session will introduce attendees to the process associated with problem identification and root cause analysis. Attendees will hear from both Center Associates, who will walk through the steps of the process and how it fits into a larger continuous improvement process. Additionally, attendees will hear from practitioners who will provide a real-world example of how they have used root cause analysis in their own work.

Webinar Recording

Session 2: Understanding the Problems for Design Innovation

Session 3: Leveraging Community for Design Innovation - Engaging stakeholders.

Monday, September 20, 2021; 3:00-4:30 p.m.

Center Staff: Carla Evans

Design innovation in educational assessment systems assumes that the design addresses stakeholder needs and concerns with the currently-implemented assessment system, as well as hopes and dreams for the future. Engaging stakeholders is a multi-pronged and ongoing endeavor that, ideally, should inform the problem statement, theory of action, prototyping evaluation, and scaling plans. In other words, stakeholders are a critical aspect of design innovation in educational assessment systems at all phases – especially with assessment systems designed to change, inform, or drive instructional practice in some way. In this session, we will discuss tensions that exist in stakeholder engagement and ways in which two educational leaders have engaged stakeholders in their unique contexts. Brian Reiter from the Hawai’i Department of Education will share about their process of stakeholder engagement in the design of the Hawai’i Comprehensive Assessment Program. Jeff Broom from Chicago Public Schools will share about stakeholder engagement in the design of Chicago Public School’s innovative accountability system.

Webinar Recording

Session 3: Leveraging Community for Design Innovation

Session 4: Planning for Design Innovation – Assessment Systems and Theory of Action.

Thursday, September 23, 2021; 1:00-2:30 p.m.

Center Staff: Nathan Dadey & Erika Landl

Articulating and interrogating the way in which a system is meant to function - and actually functions - in service to achieving a specified goal or goals is critically important. In this session, we use a theory of action framework as a tool for structuring, communicating, and implementing balanced assessment systems. We define the characteristics of a balanced assessment system, explore how they intersect with a TOA framework, and outline the focal role of assessment information when considering system design and composition.

Webinar Recording
Theory of Action Template
Interim Assessment Toolkit Tools:
Part 1: Teaching and Learning
Part 2: Using Assessments
Part 3: Assessment Inventory

Planning for Design Innovation

Session 5: Exploring the Design Innovation Process – Iteration in Assessment System Design.

Thursday, September 23, 2021; 3:00-4:30

Center Staff: Brian Gong

Guest Presenters: Alina von Davier and Gene Wilhoit

This session will explore how the design of innovative assessments can be informed by rapid prototyping, feedback, and successive revisions at all stages of the development lifecycle. Summaries will be presented as to why iteration in design is necessary, when it is productive, and why it is sometimes difficult. Advice from the real world will be offered in an engaging moderated discussion format by experts in assessment policy, assessment implementation, and assessment technical development.

Webinar Recording

Exploring the Design Innovation Process

Session 6: Exploring the IADA Innovation Process – Opportunities and Challenges. 

Part 1: Challenges

Friday, September 24, 2021; 1:00-2:30 p.m.

Center Staff: Carla Evans & Scott Marion

The Innovative Assessment Demonstration Authority under the Every Student Succeeds Act was purportedly created to provide space for states to design and implement “competency-based or other innovative assessment approaches for use in making accountability determinations” (ESSA Sec. 1204). Assessment design – innovative or not – is always a case of optimization under constraints. In this session, Scott Marion and Carla Evans, both of whom have been deeply involved in various IADA initiatives since the program was initiated, will examine the challenges and opportunities of trying to innovate state assessment systems under the federal assessment and accountability requirements (constraints). In doing so they will draw on current state examples to illustrate various approaches to the design process and offer their insights regarding the potential for innovation.

Webinar Recording
Brief Summary of IADA Programs

1. Tensions Over Multiple, Intended Uses: Informing Instruction and Summative Accountability
2. Scaling the Innovation Statewide
3. Psychometric Scaling
4. Comparability Challenges
5. Advice to States
6. Advice to the US Department of Education
7. Concluding Thoughts: Tensions, Tradeoffs, and Innovation

Exploring the IADA Innovation Process

Session 6: Exploring the IADA Innovation Process – Opportunities and Challenges. 

Part 2: Opportunities: The Outlook for Innovation: Opportunities and Challenges on the Road Ahead

Friday, September 24, 2021; 1:00-2:30 p.m.

Center Staff: Chris Domaleski

What have we learned from prominent attempts to promote innovation in assessment and accountability in recent decades, and how can we leverage this information to navigate the road ahead? Chris Domaleski will close the 2021 RILS session by reflecting on potential areas of innovation in the near future and providing insights on how the field can be better poised to take advantage of the coming opportunities. In particular, Chris will discuss the complex relationship between assessment and accountability and emphasize the importance of addressing both to promote progress.

Exploring the IADA Innovation Process