2019 Reidy Interactive Lecture Series

Improving the Selection, Use, and Evaluation of Interim Assessments

Interim Assessment Toolkit Part 1: Identifying Assessment Gaps and Needs

Interim Assessment Toolkit Part 2: Assessment Characteristics and Evidence of Quality

Thursday, September 26th

Agenda

TimeTopic
7:30amRegistration & Breakfast
Registration and breakfast will be available for attendees beginning at 7:30.
8:30amWelcome to RILS
Scott Marion, Center for Assessment

The Center for Assessment will welcome participants to the 21st annual Reidy Interactive Lecture Series and provide a brief overview of the conference.

Related resources:
Setting the Stage
Not as Easy as It Sounds: Designing a Balanced Assessment System
Moving Toward a Comprehensive Assessment System: A Framework of Considering Interim Assessments
A Tricky Balance: The Challenges and Opportunities of Balanced Systems of Assessment
A Framework for Considering Interim Assessments
Evaluating test validity: reprise and progress
RILS Agenda
Morning Day 1 Slides
8:45amSetting the Stage
Nathan Dadey, Center for Assessment
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment
Scott Marion, Center for Assessment

During this session, the Center will describe the landscape of interim assessments, provide a working definition of interim assessments for the conference, describe some common barriers to selecting and evaluating interim assessments, and describe the primary outcomes for RILS 2019.

Related resources:
A Rapid Review of Interim Assessment Use
State Interim Scan (Excel)
One Lay of the Land: Preliminary Findings from Two High Level Scans
9:45amPurposes of Interim Assessments: Small Group Feedback and Discussion
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment

Working in small groups, participants will discuss issues faced by states, districts, and schools related to how interim assessments are selected and used. The purpose of this discussion is to identify common challenges associated with selecting assessments (e.g., coherence), identifying desired assessment characteristics, and evaluating the impact and utility of interim assessment results.
10:45amBreak
11:00amState and District Case Study Discussion
Leslie Keng, Center for Assessment
Laurie Hernandez & Jessica Steinbrenner, Wyoming Department of Education
Sonya Tysdal, Weston County School District 1, WY
Lorrie Shepard, University of Colorado at Boulder

During this session state and district leaders from Wyoming will share their perspectives related to the role and purpose of interim assessments and discuss what they see as the greatest challenges that need to be addressed in order to ensure they add value. An expert discussant will provide comments and reactions.
12:00pmLunch
1:00pmThe Importance of Identifying Assessment Gaps and Needs
Juan D’Brot, Center for Assessment

During this brief session, the Center will highlight the need for a thoughtful process prior to making important decisions about the need for an interim assessment and its intended role, if any, in an existing assessment system. We will discuss the first phase of a proposed process, Identifying Gaps and Needs, which requires education leaders to articulate their vision for teaching and learning and their assessment theory of action prior to conducting an assessment search, review, or evaluation.
1:15pmDistrict Perspective: School Administrative Unit #9, New Hampshire
Carla Evans, Center for Assessment
Kadie Wilson, School Administrative Unit 9, NH

This session will highlight how a district considers the design and selection of interim assessments based on its unique context and identified information needs. The conversation will address the district’s vision, how they intend to use assessment information, and how they specified high priority assessment selection criteria.
1:45pmDiscussion and Refinement of Phase 1 of the Interim Assessment Identification and Evaluation Process: Identifying Assessment Gaps and Needs
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment

During this session we will break into groups to explore a sample district profile that illustrates the types of information a district might compile based on responses to Phase 1 of the Interim Assessment Identification and Evaluation Process. After reviewing the profile, groups will be asked to review the questions and structure of the Phase 1 tool to determine whether there is sufficient guidance and scaffolding for someone to respond to each set of questions focused on identifying assessment gaps and needs. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes. An electronic template will be established so that feedback can be captured during the discussion and shared with the large group tomorrow.
3:00pmBreak
3:15pmIntroduction to Phase 2: Identifying and Prioritizing Assessment Characteristics & Evidence of Assessment Quality
Erika Landl, Center for Assessment

During this brief session, the Center will discuss Phase 2 of the process, introduce the Assessment Characteristics Tool and discuss how it is intended to be used to support assessment selection and evaluation. The intent of this tool is to help state and district leaders clarify and articulate the specific design, administration and reporting characteristics an interim assessment must demonstrate to meet the district’s information needs in a manner that is consistent with its vision for teaching and learning.
3:35pmDiscussion and Refinement of the Assessment Characteristics Tool
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment

During this session, participants will again break into groups. Working at tables, participants will be asked to use the same scenarios considered during the previous activity and the Phase 2 tool to determine the primary characteristics and features that an assessment should demonstrate to meet the district’s goals. Each table will be asked to develop a broad profile of the type of assessment they are looking for and provide their thoughts regarding resources, guidance and business rules that would need to be developed or provided to support implementation in schools. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes. An electronic template will be established so that feedback can be captured during the discussion and shared with the large group tomorrow.
4:40pmClosing Comments and Day 2 Preview
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment
4:45pmAdjourn for Day 1
5:15pmWelcome Reception

Friday, September 27th

TimeTopic
7:30amBreakfast
Breakfast will be available for attendees beginning at 7:30.
8:30amWelcome Back and a Brief Overview of Evidence for Evaluating Technical Quality
Brian Gong, Center for Assessment

During this session, the Center will welcome the group and discuss the interplay between assessment characteristics and appropriate evidence of technical quality. We also introduce a resource developed to help districts identify and prioritize evidence of assessment quality given information collected with the Assessment Characteristics tool.

Evaluation Slides
8:45amOngoing Evaluation – Discussing the Need for a Use and Utility Evaluation Tool
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment

During this session we will make a case for the ongoing evaluation of interim assessments to ensure they are adding value and being used and interpreted as intended. We will discuss the relevance of program evaluation in the context of information use, how evaluative practices could help address implementation and unintended consequences, and the need to collect evidence that goes beyond the evaluation of the assessment itself.
9:00amState and District Perspectives: Panel and Group Discussion on the Use and Utility of Interim Assessments
Scott Marion, Center for Assessment
Marianne Perie, Measurement in Practice
Margaret Goertz, Consortium for Policy Research in Education, University of Pennsylvania
Charity Flores & Mary Williams, Indiana Department of Education
Craig Walker, Oklahoma Department of Education
Nick Meyer, Indianapolis Public Schools

This session will address the challenges associated with consequences and use of interim assessment information. Experts, states, and districts will be asked to describe their perspective on evaluating the implementation, utility, and impact of interim assessments and how operational constraints play a role. The Center will conclude this session by describing opportunities for developing a tool or framework that can help practitioners support the implementation of interim assessments.
10:30amBreak
10:45amDeveloping Specifications for a Use and Utility Tool
Juan D’Brot, Center for Assessment
Erika Landl, Center for Assessment

During this session, participants will break into groups to discuss the highest priority categories that could be used to evaluate the use of interim assessments and their associated information. Participants will be asked to consider their own experiences to help address evaluating the capacity and behaviors of people who are using interim assessment information to help avoid unintended consequences of misuse. The Center will take this feedback to develop a Use and Utility Tool, which will be shared with participants and others freely on our website. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes.  An electronic template will be established so that feedback can be captured during the discussion.
11:45amNext Steps and Closing Comments
Chris Domaleski, Center for Assessment
12:00pmConclude for Lunch

Share: