Registration and Breakfast

Welcome to RILS
The Center for Assessment will welcome participants to the 21st annual Reidy Interactive Lecture Series and provide a brief overview of the conference.

PRESENTER: Scott Marion (Center for Assessment)

Setting the Stage
During this session, the Center will describe the landscape of interim assessments, provide a working definition of interim assessments for the conference, describe some common barriers to selecting and evaluating interim assessments, and describe the primary outcomes for RILS 2019.

PRESENTERS: Nathan Dadey (Center for Assessment), Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment), Scott Marion (Center for Assessment)

PRESENTATIONS:
Interim Lay of the Land: Preliminary Findings from Two High Level Scans

RESOURCES:
Designing Balanced Assessment Systems (Chatergoon and Marion, 2016)
A Tricky Balance (Marion, et. al, 2019)
Interim Assessment Use (Dadey and Diggs, 2019)
Scan of Interim Assessments Spreadsheet (Dadey and Diggs, 2019)
Defining Interim Assessment (Perie, Marion, Gong, 2007)
Defining Interim Assessment (Perie, Marion, Gong, 2009)

Purposes of Interim Assessments: Small Group Feedback and Discussion
Working in small groups, participants will discuss issues faced by states, districts, and schools related to how interim assessments are selected and used. The purpose of this discussion is to identify common challenges associated with selecting assessments (e.g., coherence), identifying desired assessment characteristics, and evaluating the impact and utility of interim assessment results.

PRESENTERS: Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment)

PRESENTATION:
Morning_Day1_Slides

Break

State and District Case Study Discussion
During this session state and district leaders from Wyoming will share their perspectives related to the role and purpose of interim assessments and discuss what they see as the greatest challenges that need to be addressed in order to ensure they add value. An expert discussant will provide comments and reactions.

PRESENTERS: Leslie Keng (Center for Assessment), Laurie Hernandez & Jessica Steinbrenner (Wyoming Department of Education), Sonya Tysdal (Weston County School District 1, WY), Scott Marion (Center for Assessment)

PRESENTATIONS:
WY_Intro_Slides
WY_State&District_Slides

Lunch

The Importance of Identifying Assessment Gaps and Needs
During this brief session, the Center will highlight the need for a thoughtful process prior to making important decisions about the need for an interim assessment and its intended role, if any, in an existing assessment system. We will discuss the first phase of a proposed process, Identifying Gaps and Needs, which requires education leaders to articulate their vision for teaching and learning and their assessment theory of action prior to conducting an assessment search, review, or evaluation.

PRESENTER: Juan D'Brot (Center for Assessment)

PRESENTATION:
Gaps&Needs_Intro_Slides

District Perspective: School Administrative Unit #9, New Hampshire
This session will highlight how a district considers the design and selection of interim assessments based on its unique context and identified information needs. The conversation will address the district’s vision, how they intend to use assessment information, and how they specified high priority assessment selection criteria.

PRESENTERS: Carla Evans (Center for Assessment), Kadie Wilson (School Administrative Unit 9, NH)

Discussion and Refinement of Phase 1 of the Interim Assessment Identification and Evaluation Process: Identifying Assessment Gaps and Needs
During this session we will break into groups to explore a sample district profile that illustrates the types of information a district might compile based on responses to Phase 1 of the Interim Assessment Identification and Evaluation Process. After reviewing the profile, groups will be asked to review the questions and structure of the Phase 1 tool to determine whether there is sufficient guidance and scaffolding for someone to respond to each set of questions focused on identifying assessment gaps and needs. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes. An electronic template will be established so that feedback can be captured during the discussion and shared with the large group tomorrow.

PRESENTERS: Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment)

Break

Introduction to Phase 2: Identifying and Prioritizing Assessment Characteristics & Evidence of Assessment Quality
During this brief session, the Center will discuss Phase 2 of the process, introduce the Assessment Characteristics Tool and discuss how it is intended to be used to support assessment selection and evaluation. The intent of this tool is to help state and district leaders clarify and articulate the specific design, administration and reporting characteristics an interim assessment must demonstrate to meet the district’s information needs in a manner that is consistent with its vision for teaching and learning.

PRESENTER: Erika Landl (Center for Assessment)

PRESENTATION:
Assess_Char_Intro_Slides

Discussion and Refinement of the Assessment Characteristics Tool
During this session, participants will again break into groups. Working at tables, participants will be asked to use the same scenarios considered during the previous activity and the Phase 2 tool to determine the primary characteristics and features that an assessment should demonstrate to meet the district’s goals. Each table will be asked to develop a broad profile of the type of assessment they are looking for and provide their thoughts regarding resources, guidance and business rules that would need to be developed or provided to support implementation in schools. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes. An electronic template will be established so that feedback can be captured during the discussion and shared with the large group tomorrow.

PRESENTERS: Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment)

Closing Comments and Day 2 Preview

PRESENTERS: Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment)

Adjourn for Day 1

Welcome Reception

Breakfast

Welcome Back and a Brief Overview of Evidence for Evaluating Technical Quality
During this session, the Center will welcome the group and discuss the interplay between assessment characteristics and appropriate evidence of technical quality. We also introduce a resource developed to help districts identify and prioritize evidence of assessment quality given information collected with the Assessment Characteristics tool.

PRESENTER: Brian Gong (Center for Assessment)

PRESENTATION:
Gong_Eval_Slides

RESOURCES:
Matching Instructional Uses with Design (Gong, 2019)
Evaluating test validity (Shepard, 2016)

Ongoing Evaluation – Discussing the Need for a Use and Utility Evaluation Tool
During this session we will make a case for the ongoing evaluation of interim assessments to ensure they are adding value and being used and interpreted as intended. We will discuss the relevance of program evaluation in the context of information use, how evaluative practices could help address implementation and unintended consequences, and the need to collect evidence that goes beyond the evaluation of the assessment itself.

PRESENTERS: Juan D'Brot (Center for Assessment), Erika Landl (Center for Assessment)

PRESENTATION:
Evaluation_Intro_Slides

State and District Perspectives: Panel and Group Discussion on the Use and Utility of Interim Assessments
This session will address the challenges associated with consequences and use of interim assessment information. Experts, states, and districts will be asked to describe their perspective on evaluating the implementation, utility, and impact of interim assessments and how operational constraints play a role. The Center will conclude this session by describing opportunities for developing a tool or framework that can help practitioners support the implementation of interim assessments.

PRESENTERS: Scott Marion (Center for Assessment), Marianne Perie (Measurement in Practice), Margaret Goertz (Consortium for Policy Research in Education, University of Pennsylvania), Charity Flores & Mary Williams (Indiana Department of Education), Craig Walker (Oklahoma Department of Education), Nick Meyer (Indianapolis Public Schools)

PRESENTATIONS:
Use_Panel_Slides

Break

Developing Specifications for a Use and Utility Tool
During this session, participants will break into groups to discuss the highest priority categories that could be used to evaluate the use of interim assessments and their associated information. Participants will be asked to consider their own experiences to help address evaluating the capacity and behaviors of people who are using interim assessment information to help avoid unintended consequences of misuse. The Center will take this feedback to develop a Use and Utility Tool, which will be shared with participants and others freely on our website. Each group will be partnered with 1-2 Center staff who will help facilitate the discussion and take notes. An electronic template will be established so that feedback can be captured during the discussion.

PRESENTERS: Juan D’Brot (Center for Assessment), Erika Landl (Center for Assessment)

PRESENTATIONS:
Eval_Considerations_Slides

Next Steps and Closing Comments

PRESENTER: Chris Domaleski (Center for Assessment)

Conclude for Lunch