What We Can Learn From Bill Belichick About Assessment Literacy
Bill Belichick is a famous football coach known for his dry humor and non-answer answers to questions from the media. Coach Belichick holds many coaching records; including a record five Super Bowls as a head coach. One of his more famous quotes is “Do your job” and a big part of his success as a coach is preparing players to do their job well. You might not think Coach Belichick has a lot to offer in a discussion about assessment literacy, but I disagree.
Belichick has perfected the art of situational football. Situational football is the application of football knowledge and skills within the context of real-time decisions that need to be made during games. What good is it to know all the rules of the game and have the best plays, but not be able to manage the complex decisions and scenarios that arise during a game? Situational football addresses the complexity of integrating knowledge and skill into making the best decisions possible in certain circumstances.
Assessment literacy should operate in a similar fashion. Assessment literacy for educators is the knowledge and skills associated with designing, selecting, interpreting, and using high-quality assessments to improve student learning and to serve other important educational and policy purposes. While many have produced lists of discrete assessment literacy knowledge and skills for different stakeholder groups (e.g., parents, teachers, school/district leaders, state policymakers), assessment literacy is fundamentally a situated activity. Each stakeholder is required to use their assessment knowledge and skills within the context of real-world and real-time problems and decisions in order to do their jobs well.
Situational assessment literacy may begin with asking two simple questions: (1) What decisions do teachers, school/district leaders, and state policymakers make about designing, selecting, interpreting, and/or using assessments? (2) What knowledge and skills do these stakeholder groups need in order to make the best possible decisions about assessments in their context? Let me give you some examples:
- A teacher is planning a unit on fractions for grade 4 math. He is trying to figure out how to assess student mastery before, during, and after instruction.
- A principal is trying to interpret and use state assessment results for program evaluation purposes.
- An Assistant Superintendent just received a handful of brochures in the mail trying to sell interim/benchmark assessments and other assessment products. She is trying to figure out if her district should buy one of the interim/benchmark assessments.
- State policymakers are considering changing the state assessment system and creating an RFP for a new test vendor. They are unsure of the implications of this decision or how to go about making this decision.
What advice would you give this teacher, school leader, or state policymaker? What assessment literacy concept(s) did you draw on and apply to this situation? Let me illustrate what I mean using the teacher example.
First, I would inquire why the teacher intends to assess student ‘mastery’ and how the assessment evidence is intended to be used. Decisions with high-stakes consequences require different sufficiency considerations related to the quantity and quality of evidence. Assuming typical classroom use, my main advice to the teacher would be to start at the end and backward design. That is, to start by defining what it is that he wants students to know and be able to do with fractions in grade 4 math. This is not just the standards, but the big ideas of the discipline, including essential questions and enduring understandings. Then I’d advise the teacher to do a thought experiment to consider what would count as acceptable evidence that the students have indeed learned what was intended. It is only at this point that I would advise the teacher to then design a pre-assessment, formative checks for understanding or misconceptions, and a summative performance task or test to elicit the desired evidence.
The assessment literacy concepts I drew on to advise the teacher included a principled assessment design approach (e.g., Mislevy, Steinberg, Almond, 2003; NRC, 2001) to designing assessments. Principled assessment design focuses on the idea that assessments should be designed from the inferences they are intended to support and connects a focus on learning with assessment design. This type of approach ensures that any and all designed assessments will indeed measure important and meaningful learning goals and provide appropriate evidence that the student has (or has not) learned what was intended.
In a recent paper prepared for the Center’s 20th anniversary conference, my colleagues and I argued that many have been working to improve assessment literacy for years, but there appears to be only limited growth in educators’ assessment literacy. Perhaps this means we have been going about this in unproductive ways? Using Belichick as a guide, perhaps we should consider helping teachers, school/district leaders, and state policymakers develop the art of situational assessment literacy. In practice this may look a lot like what Coach Belichick does with his players: helping those involved anticipate the likely situations and decisions they will encounter and practicing how to respond and prepare in order to make the best possible decisions in their given context. In our paper, we humbly offer suggestions for beginning to address the challenges related to situational assessment literacy for these three stakeholder groups.
Click here to access the full paper and scroll to the section on assessment literacy that begins on page 20.