Past Conferences
In an effort to provide states with concrete technical guidelines addressing new forms of assessment, newly mandated accountability systems, and an increased emphasis on student achievement, the Center for Assessment established the Edward F. Reidy, Jr. Interactive Lecture Series (RILS). The goal of the conference is to bring together educational researchers, policy makers, and practitioners to exchange information and discuss and generate practical recommendations. The first annual conference was held in 1999. Each year the conference focuses on a timely topic.
2009 Conference |2008 Conference | 2007 Conference
| 2006 Conference | 2005 Conference
| 2004 Conference2003 Conference | 2002 Conference |
2001 Conference | 2000 Conference | 1999 Conference
2009 Conference: October 1-2, 2009 Portsmouth, NH
Next Generation Assessment and Accountability Systems
The focus of the 2009 RILS conference was on Next Generation Assessment and Accountability Systems. Our starting point, however, was consideration of the fundamental goals of public education that those assessment and accountability systems are intended to support. Only with an understanding of what we expect from public education can we design appropriate assessment and accountability systems that are aligned with those goals. Looking ahead toward an image of where public education should and can be in fifteen years, presenters, panelists, and attendees discussed the type of assessment and accountability systems necessary to support that education system and the steps needed to begin moving from our current assessment and accountability systems designed to meet the requirements of NCLB to the next generation of assessment and accountability systems. The conference was organized into half-day sessions focused on the three topics of Public Education, Assessment, and Accountability Systems. Each session included a keynote address, panel presentations and discussion, and facilitated discussion among presenters and other conference attendees.
Session I:
Session II:
Session III:
2008 Conference: September 25-26, 2008 Portsmouth, NH
Validating Assessment and Accountability Programs
The 2008 RILS will bridge theory and practice in focusing on how to validate
assessment and accountability programs building on advances in the field over
the past several years. The program will be organized around three major topics:
Validation of assessments and assessment systems
Validation of accountability decisions and accountability systems
Validating assessments and accountability for special populations
Session I:
Session II:
Moderated Panel: Key considerations for different types of validation efforts
Session III:
Session IV:
Session V:
2007 Conference: September 27-28, 2007 Nashua, NH
English Language Learner Assessment and Accountability:
Critical considerations for Design and Implementation
There are significant challenges to appropriately assessing and including
in accountability students for whom English is not their native language. This
year's RILS conference featured thought-provoking presentations and discussions
of the challenges and state-of-the-art research and practice on how to understand
and deal more validly with Title I and Title III requirements. Presentations
provided frameworks for defining what really matters for assessment and accountability
among all the factors that make the EL population unique. Guidance was provided
that outlined the specific requirements and how to address them for Title I
and Title III, with an eye for efficiencies throughout the assessment development
and use cycle. Research was provided and discussed about subgroup differences,
empirical growth rates, and accommodations effectiveness mediated by ELL student
characteristics. As usual for RILS, there was consistent attention to the policy
implications of these presentations.
Session I:
Setting the Stage: English Language Learner Assessment and Accountability
-Stanley Rabinowitz, WestEd
Session II:
ELL Assessment - Technical requirements for the assessment of ELL students
- Gary Cook (WCER)
Slides (77KB) Handout
(608KB) Handout (140KB) Handout
(402KB)
- Edynn Sato (WestEd)
Handout (434KB)
Session III:
ELL Assessment - Access and accommodations
- Charlene Rivera (Center for Equity and Excellence in Education)
- Maria Pennock-Roman (MPR Psychometric & Statistical Services)
Paper (1,068KB)
Session IV:
ELL Accountability
- Stanley Rabinowitz (WestEd)
- Scott Marion (Center for Assessment)
Slides (156KB)
- Ted Rebarber (Accountability Works)
- Shelda Hale (Kentucky Department of Education)
Handout (97KB)
Session V:
A Framework of English Language Proficiency Standards and Assessments (draft
being developed for USED)
- Edynn Sato (WestEd)
2006 Conference: October 5-6, 2006 Nashua, NH
Comprehensive Assessment Systems to Improve Student
Learning
Formative, interim, and large-scale assessments all have a place as components
of a comprehensive assessment system to help improve teaching and learning in
the classroom. This was the subject of the exciting interactions among the presenters
and participants at this year's RILS conference. The sessions focused on the
types of assessment activities being offered and we explored the role that each
can play to promote student achievement. The importance of articulating a clear
view of what should be learned and how students come to learn it-organized as
clear developmental targets or learning progressions-provided a foundational
presentation for assessment design sessions. Subsequent presentations focused
on defining, illustrating, and providing guidance for evaluating formative,
interim/benchmark, and large-scale assessments and how each contributes to creating
a comprehensive assessment system for fostering student learning. Finally, along
with outstanding leaders who are creating such comprehensive assessment systems
at the state and district levels, we discussed key policy issues and offered
insights into how to successfully address the challenges of implementing these
systems in their various settings.
Session I:
Overview of Comprehensive Assessment Systems: Yes, the Large-Scale Assessment
Matters
-Stanley Rabinowitz, WestEd
Slides (41KB)
Session II:
Establishing Learning Goals
-Brian Gong, Center for Assessment
Slides (76KB)
Session III:
Linking Formative Assessment Approaches to Instructional Decisions
-Karin Hess, Center for Assessment
Slides (121KB)
Session IV:
Design Considerations for Interim/Benchmark Assessments
-Marianne Perie, Center for Assessment
Slides (142KB)
Session V:
Formative and Benchmark Assessment Applications
-Tim Wiley, Eduventures, Inc.
Slides (179KB)
Session VI:
The State and District Role: Supporting Professional Development and
Building
Policy Structures (Implementation Strategies)
-Scott Marion, Center for Assessment
Slides (88KB)
Session VII:
Implementation Strategies for States and Districts
-Bob Stanton, Lamoille South Supervisory Union, VT
Slides (536KB)
2005 Conference: September 29-30, 2005 Nashua, NH
Wrestling With High School Assessment and Accountability
Our 7th annual conference combined beautiful weather in a tranquil New England autumn with in-depth discussion of one of today's most salient topics in assessment.
Presenters were drawn from WestEd (co-sponsor of the RILS conference), HumRRO, SREB, and the Center. The conference also provided a forum for wide-ranging and lively discussion that challenged assumptions and probed the topics deeply.
The sessions provided articulate conceptual frameworks, useful overviews of the field, a range of specific examples, and concrete recommendations regarding high school assessment and accountability. The differences between high school purposes and earlier grades was emphasized, along with validity concerns when designing assessments that range from end-of-course exams to graduation survey tests to college entrance exams. Discussions of high school accountability similarly emphasized the unique condition of high schools, including the need for comprehensive systems with distributed responsibilities and richer measures.
Session I:
Overview of High School Goals and Assessments
-Mark Musick, SREB
-Laura Slover, discussant, Achieve
Slides (278KB)
Session II:
Technical Considerations for Assessment
-Lauress Wise,
HumRRO
Slides (315KB)
Session III:
High School Accountability: A Framework
-Scott Marion, Center for Assessment
Slides (2.27MB)
-Linda Mabry, discussant,
Univ. of WA, Vancouver
Paper (35KB)
Session IV:
High School Accountability: A Longer-term View
-Brian Gong,
Center for Assessment
Slides (1.77MB)
2004 Conference: October 7-8, 2004 Nashua, NH
Incorporating Measures of Student Growth Into State Accountability Systems
The sessions provided concrete guidance on how to create content standards that clearly embody student development over time, and how to check whether the assessments are properly aligned. Different ways to measure student growth were presented, from simple to highly sophisticated multilevel models. Key decisions were identified for states to consider when designing a school accountability system that incorporates student growth, and a detailed example was provided of how that could be done.
Session I:
Validity Considerations When Measuring Growth
-Scott Marion, Center for Assessment
Slides (494K)
Session II:
Vertically-Articulated Content Standards, Test Specifications, and Score Scales
-Lauress Wise, HumRRO
Slides (350K)
-Karin Hess, Center for Assessment
Slides (1MB)
-Stanley Rabinowitz, WestEd
Slides (98K)
Session III:
Technical Considerations When Measuring Student Growth
-Pete Goldschmidt, CRESST
Slides (784K)
Session IV:
Incorporating Growth Measurements in Statewide Accountability Systems
-Richard Hill, Center for Assessment
Slides (227K)
-Brian Gong, Center for Assessment
Slides (328K)
2003 Conference: October 9-10, 2003,
Nashua, NH
Improving the Validity of States' Standards-Based Assessment and Accountability Systems
States will likely face substantive and legal challenges to the school/district classification decisions made as a result of implementing their No Child Left Behind (NCLB) accountability systems. This session is designed to assist state leaders in identifying particular threats to the legal defensibility of their system and to offer a framework and particular suggestions for implementing a more valid, and therefore more defensible, system.
Session I:
Building Legally Defensible Accountability Systems
-Scott Palmer, Nixon & Peabody, Inc.
Slides (400K)
-Richard Hill,
Center for Assessment
Slides (938K)
Session II:
Evaluating the Validity of State Accountability Systems
-Scott Marion & Brian Gong, Center for Assessment
Slides (1MB)
Session III:
Design Considerations for Building Out State Assessment Systems
-Stanley Rabinowitz, WestEd
Slides (69.8K)
-Marge Petit, Center for Assessment
Slides (310KB)
2002 Conference: October 10-11, 2002, Nashua, NH
Implementing the NCLB Act: Alignment, Reliability, Rationality
States and schools throughout the country are scrambling to implement the assessment and accountability components of the recently enacted federal No Child Left Behind Act (NCLB).
Many are struggling to develop strategies that blend hard-fought existing program components and successes with the letter and spirit of the Act's requirements. All are attempting to develop reliable, valid, and rational programs and practices that identify and support low achieving schools and students.
Session I:
Reliable accountability design decisions and NCLB:
What to do, and what not to do
-Richard Hill,
Center for Assessment
Slides (137K)
Session II:
How states are implementing the assessment & accountability requirements of NCLB
-Stanley Rabinowitz,
WestEd
Slides (174K)
Session III:
Aligning standards, assessments, and NCLB:
Current state of the art
-Sri Ananda, WestEd
Slides (183K)
-Marge Petit & Brian Gong, Center for Assessment
Slides (1.7MB)
2001 Conference: October 4-5, 2001,
Nashua, NH
Implementing an Accountability System that Improves Schools
More states are implementing accountability systems, each hoping for real improvements in student achievement. However, good intentions can often be derailed if states establish the wrong criteria for "quality" or "improvement," if they set the bar too high or too low, or if they fail to provide support for identified schools. Professionals in the field need concrete technical guidelines to address these issues.
Session I:
Defining the Correct Criteria for "Quality" or "Improvement"
-Richard Hill,
Center for Assessment
Slides (344K)
Session II:
How much improvement is possible?
-Brian Gong and Richard Tappan,
Center for Assessment
Slides (3.1MB)
Session III: Providing assistance to schools that need it
-Bill Schafer,
University of Maryland
Paper (255K)
2000 Conference: October 5-6, 2000,
Nashua, NH
Technical Issues Affecting State Accountability Systems
In response to the demand for educational accountability and greater student achievement, many state legislatures have enacted measures that call for formal, data-driven accountability systems. Often tied to rewards and sanctions, the systems pose great challenges to schools and districts. Practitioners and policy makers responsible for the design and implementation of such systems must ensure that they are educationally sound and technically defensible.
Session I: Reliability
To what extent would accountability labels and judgments about students, teachers, schools and districts be replicated if made again?
-Richard Hill, Center for Assessment
Paper (947K)
Session II: Validity
To what extent does the accountability system encourage people to do the things you want them to do?
-Eva Baker,
CRESST and UCLA
Slides (84K)
Session III: Comparability of Populations and Scores
When comparisons are made over time, how do you know the groups being compared are comparable?
-Brian Gong, Center for Assessment
1999 Conference: October 14-15, 1999, Warwick, RI
Establishing Standards for Benchmarking and Documenting the Quality of State Assessment Programs
State legislatures in record numbers are enacting laws that mandate new testing programs. Many assessment programs are not realizing the greatest positive impact on student achievement. In this charged climate, many practitioners feel they have been presented with the daunting tasks of meeting the educational demands for relevance and responsiveness, the political demands for accountability, and the technical requirements for quality and legal defensibility.
Session I: Reliability
Critical reliability statistics to compute
-Ed Haertel, Stanford University
Session II: Validity
Analyses needed to support validity of inferences of an assessment program
-Suzanne Lane, University of Pittsburgh
Paper (218K)
Session III: Standard Setting
Steps and methods for setting standards for performance assessment
-Ron Hambleton, University of Massachusetts
Paper (192K)
Session IV: Technical Reports
What information must be included in technical reports?
-Brian Gong, Center for Assessment
Paper
| 
