A New Framework for Universal Design of Assessment Systems

Assessment Systems
Apr 03, 2024

The Case for Reimagining Universal Design

You may have noticed that universal design is having a bit of a moment in education circles. On March 18, CAST released a preview of Universal Design for Learning 3.0, and in February, the National Center for Education Outcomes (NCEO) published a comprehensive review of research on universal design in an assessment context. We’re adding to that moment: We’re releasing a new universal design framework of our own, to support assessment in ways the existing frameworks don’t. But first, a little background.

The original principles were developed by Ron Mace and refined by the Center for Universal Design at North Carolina State University. Since then, two key developments applied these ideas to K-12 education. CAST developed Universal Design for Learning, originally to address accessibility of technology in learning environments, and NCEO authored Universal Design of Assessments to apply Mace’s principles to large-scale assessments. These frameworks and their core principles are summarized in the table below:

FrameworkCore Principles
Universal Design (Center for Universal Design)1. Equitable use
2. Flexibility in use
3. Simple and intuitive use
4. Perceptible information
5. Tolerance for error
6. Low physical effort
7. Size and space for approach and use
Universal Design for Learning (CAST)1. Multiple means of engagement
2. Multiple means of representation
3. Multiple means of response
Universal Design of Assessments (NCEO)1. Inclusive assessment population
2. Precisely defined constructs
3. Accessible and non-biased items
4. Amenable to accommodations
5. Simple, clear, and intuitive instructions and procedures
6. Maximum readability and comprehensibility
7. Maximum legibility

So What’s Wrong With Universal Design?

Despite the existence of several frameworks, the Standards for Educational and Psychological Testing—the canon of standardized testing—and laws requiring the use of some variant of universal design (including the Every Student Succeeds Act), definitions of universal design are overly broad. This lack of clarity makes universal design, as applied to assessment, difficult to operationally define, implement, and evaluate.

For example, the most recent program review of the Innovative Assessment Demonstration Authority (IADA) indicated that universal design was one of the few criteria every state met. Yet that review used a broad definition in giving states credit for meeting the universal design criterion. They met that mark if “the totality of the items developed strives to give all students equal opportunity to access test items and show what they know.”

Limitations of Each Universal Design Framework

We have advocated for universal design for many years, so it may surprise you to learn that we find all of the existing frameworks lacking in some important ways when it comes to designing assessment systems. We have a bit of a “Goldilocks and the Three Bears” problem: none of them are “just right.”

North Carolina State’s Universal Design framework, designed for architecture and products, is not specific enough to an assessment context to provide clear guidance to test developers. While the NCEO Universal Design of Assessments framework does provide specific assessment guidance, it is too narrowly focused on student and item interactions (neglecting the larger purpose of education and assessment).

CAST’s Universal Design for Learning (UDL) could serve as a middle ground, but it doesn’t adequately address approaches to large-scale assessment, despite assessment’s importance in teaching and learning. In addition, the prior version of UDL is frequently confused with the unsupported theory of “learning styles” and has faced recent criticism for relying on an overgeneralization of neuroscience research  and a lack of research to support efficacy. We were pleased to see that UDL 3.0 appears to address our concerns about the research basis, but our concerns about application to large-scale assessment remain.

Another limitation of all the frameworks is that they focus primarily on design for the students and overlook all other people in the educational system. Finally, none of these frameworks harmonize with existing standards and regulations for accessibility, such as the Standards for Educational and Psychological Testing or the W3C Web Content Accessibility Guidelines.

We Need Universal Design of Assessment Systems

Despite the limitations we’ve described, we recognize the incredible advances the “UD community” has made to ensure that universal design requirements permeate the laws and design requirements that extend beyond educational assessment. So, instead of abandoning these efforts, we have spent the last year and a half developing a new framework for universal design.

Our framework, Universal Design of Assessment Systems, or UDAS, is rooted in Mace’s original universal design principles from the field of architecture. But it also incorporates (1) contemporary perspectives on neurodiversity and culturally and linguistically relevant practices, (2) psychological theory of motivation and engagement, and (3) technical requirements for assessment use cases.

For example, we translate the original Universal Design principle of “safe use” into an assessment context that emphasizes designs that aim to mitigate unintended consequences. This is essential given the history of harm caused by the misuse of high-stakes tests.

We built a coherent framework that extends across all types of assessments (formative, diagnostic, interim, and summative) because systems design acknowledges the needs of all humans in the system rather than limiting guidance solely to student-item interactions. We don’t have enough room in this blog to fully describe our approach, but its core three principles are:

  • Principle 1: Assessment systems must meet basic human needs, honor test takers as learners, and fulfill the ultimate goals of education
  • Principle 2: Design assessments for flexibility and learner variability
  • Principle 3: Promote equity and take action to mitigate unintended consequences

If you would like to learn more, we will present a paper on our new framework, Universal Design of Assessment Systems (UDAS), on Saturday, April 13 at the annual meeting of the National Council on Measurement in Education (NCME). Please join us. If you can’t make it to NCME, don’t worry—we will update this post with a link to the full paper as soon as it’s published.

Meagan Karvonen is the director of Accessible Teaching, Learning, and Assessment Systems (ATLAS) at the University of Kansas.

Share: