Held each fall since 1999, the Reidy Interactive Lecture Series (RILS) provides an opportunity for a diverse group of participants to discuss timely and important topics in assessment and accountability. This year, RILS will continue as a no-cost, virtual offering that we hope will sustain the great attendance and participation we saw last year. RILS 2021 will comprise six sessions on four different days. Details of each session and registration are as follows:
Session 1: Introduction to Design Innovation – Design and innovation in the creation of educational assessment systems.
Wednesday, September 15, 2021; 1:00-2:30 p.m.
Guest Presenter: Charles DePascale
Drawing on his own experiences with innovation and educational assessment over the past six decades, Charlie will offer a brief look at the history of innovation in large-scale educational assessment – which inventions became innovations and which became just the latest fad? When was innovation in educational assessment driven by Design Thinking and what difference did that make? What separates recent attempts at innovation in educational assessment from just about every successful innovation?
Session 2: Understanding the Problems for Design Innovation - Root Cause Analysis.
Monday, September 20, 2021; 1:00-2:30 p.m.
Center Staff: Juan D’Brot and Chris Brandt
Designing a balanced assessment system that meets the various needs of stakeholders requires both an understanding of the context as well as the problem that needs to be solved through the redesign of the assessment system. In order to support a redesign, the problems must be well-specified and focused, which may require breaking apart the larger problems into manageable components, supporting more effective root cause analyses, and helping to specify a theory of action. This session will introduce attendees to the process associated with problem identification and root cause analysis. Attendees will hear from both Center Associates, who will walk through the steps of the process and how it fits into a larger continuous improvement process. Additionally, attendees will hear from practitioners who will provide a real-world example of how they have used root cause analysis in their own work.
Session 3: Leveraging Community for Design Innovation - Engaging stakeholders.
Monday, September 20, 2021; 3:00-4:30 p.m.
Center Staff: Carla Evans
Design innovation in educational assessment systems assumes that the design addresses stakeholder needs and concerns with the currently-implemented assessment system, as well as hopes and dreams for the future. Engaging stakeholders is a multi-pronged and ongoing endeavor that, ideally, should inform the problem statement, theory of action, prototyping evaluation, and scaling plans. In other words, stakeholders are a critical aspect of design innovation in educational assessment systems at all phases – especially with assessment systems designed to change, inform, or drive instructional practice in some way. In this session, we will discuss tensions that exist in stakeholder engagement and ways in which two educational leaders have engaged stakeholders in their unique contexts. Brian Reiter from the Hawai’i Department of Education will share about their process of stakeholder engagement in the design of the Hawai’i Comprehensive Assessment Program. Jeff Broom from Chicago Public Schools will share about stakeholder engagement in the design of Chicago Public School’s innovative accountability system.
Session 4: Planning for Design Innovation – Assessment Systems and Theory of Action.
Thursday, September 23, 2021; 1:00-2:30 p.m.
Center Staff: Nathan Dadey & Erika Landl
Articulating and interrogating the way in which a system is meant to function - and actually functions - in service to achieving a specified goal or goals is critically important. In this session, we use a theory of action framework as a tool for structuring, communicating, and implementing balanced assessment systems. We define the characteristics of a balanced assessment system, explore how they intersect with a TOA framework, and outline the focal role of assessment information when considering system design and composition.
Session 5: Exploring the Design Innovation Process – Iteration in Assessment System Design.
Thursday, September 23, 2021; 3:00-4:30
Center Staff: Brian Gong
Guest Presenters: Alina von Davier and Gene Wilhoit
This session will explore how the design of innovative assessments can be informed by rapid prototyping, feedback, and successive revisions at all stages of the development lifecycle. Summaries will be presented as to why iteration in design is necessary, when it is productive, and why it is sometimes difficult. Advice from the real world will be offered in an engaging moderated discussion format by experts in assessment policy, assessment implementation, and assessment technical development.
Session 6: Exploring the IADA Innovation Process – Opportunities and Challenges.
Part 1: Challenges
Friday, September 24, 2021; 1:00-2:30 p.m.
Center Staff: Carla Evans & Scott Marion
The Innovative Assessment Demonstration Authority under the Every Student Succeeds Act was purportedly created to provide space for states to design and implement “competency-based or other innovative assessment approaches for use in making accountability determinations” (ESSA Sec. 1204). Assessment design – innovative or not – is always a case of optimization under constraints. In this session, Scott Marion and Carla Evans, both of whom have been deeply involved in various IADA initiatives since the program was initiated, will examine the challenges and opportunities of trying to innovate state assessment systems under the federal assessment and accountability requirements (constraints). In doing so they will draw on current state examples to illustrate various approaches to the design process and offer their insights regarding the potential for innovation.
Materials:
Brief Summary of IADA Programs
Videos:
Tensions Over Multiple, Intended Uses: Informing Instruction and Summative Accountability
Scaling the Innovation Statewide
Psychometric Scaling
Comparability Challenges
Advice to States
Advice to the US Department of Education
Concluding Thoughts: Tensions, Tradeoffs, and Innovation
Session 6: Exploring the IADA Innovation Process – Opportunities and Challenges.
Part 2: Opportunities: The Outlook for Innovation: Opportunities and Challenges on the Road Ahead
Friday, September 24, 2021; 1:00-2:30 p.m.
Center Staff: Chris Domaleski
What have we learned from prominent attempts to promote innovation in assessment and accountability in recent decades, and how can we leverage this information to navigate the road ahead? Chris Domaleski will close the 2021 RILS session by reflecting on potential areas of innovation in the near future and providing insights on how the field can be better poised to take advantage of the coming opportunities. In particular, Chris will discuss the complex relationship between assessment and accountability and emphasize the importance of addressing both to promote progress.
