Enhanced Score Reporting for a Game-Based Assessment

Game-Based Assessment Can be Fun for Students: Can We Make Its Use More Informative for Teachers?

This is the fifth post by one of our 2021 summer interns based on their project and the assessment and accountability issues they are addressing this summer with their Center mentors. Ella Anghel, from Boston College, is working with Nathan Dadey and Will Lorié in examining data from a game-based assessment to determine an approach to summarizing the data to provide useful information to teachers.

My psychology professors used to say that a lot can be learned about a child simply from watching them play. But only in recent years have game-based assessments become more formal, with researchers developing large-scale assessments designed to capture the target constructs while being fun and engaging for the student. As one can imagine, game-based assessments generate a lot of data, and making sense of all of that data can be tricky, especially for busy teachers working with groups of students. The purpose of my internship project involved finding a way to summarize assessment data to provide information that is useful for teachers.

Educators and other professionals have long attempted to make teaching and assessments more engaging by incorporating playful elements. Some existing games have been shown to successfully measure a variety of skills of students of all ages (e.g., Hautala et al., 2020; Kim & Ifenthaler, 2019). One of these games was the focus of my internship. In this game-based assessment, the players enter a village full of colorful characters and help the characters develop their village by solving math and reading problems. First- and second-grade teachers use this game to assess the students’ understanding immediately after learning something new. The game collects data on students’ responses and actions, which are then reported to the teacher. This information can help teachers focus on more difficult concepts or support specific students who need more instruction. 

Information Overload: Applying a Person-Centered Approach to a Game-Based Assessment 

The purpose of my internship project involved finding ways to summarize assessment information for teachers. We focused on a game designed to measure reading fluency. This game consisted of ten levels, with more advanced levels representing more complex words to read and understand. A student who plays the game starts at the easiest level; they advance to the next level if they get two or fewer of the items wrong (out of ten). Otherwise, the game restarts. Each level is associated with a specific early literacy skill.

The students seem to love this game and are motivated to reach the hardest level – many of them play the game repeatedly, even within the same play session. This practice, of course, exacerbates the problem of abundant data, because the teachers get reports on all of their students’ answers, as well as the time it takes each student to answer each item (i.e., response time). How can anyone make sense of all of these data? 

To address this challenge, we propose to replace the data on student responses to individual items with a person-centered approach to reporting the assessment results. Person-centered approaches do not focus on individual variables (e.g., responses to a specific item), but on types or profiles of people. This approach usually generates a small number of distinct groups of people characterized by a set of variables. 

In the current case, instead of overwhelming teachers with information about their students’ answers and response times on all of the items they have attempted, we ask: Is it possible to group students into categories that summarize all of this information? That type of profile information would be much easier to understand and could save teachers the time and effort required to decipher each data point. 

Using this approach, we were able to identify three distinct groups of students using data from 2,255 students who took the assessment for the first time in September 2020. 

• The first and largest group was the least successful, with about half of the students advancing to the second level and very few advancing past level five. 
• The second group was able to get to level three, where they started making more mistakes and struggled to pass level seven. 
• The last and smallest group consisted of students who got to the most advanced levels, with some of them even beating the game on their first try. 

What Does it All Mean for Teachers?

These profiles clearly show that some students have already mastered the skills covered by the game at the beginning of the school year and were able to finish the game relatively quickly. This outcome is even more impressive when considering that these students just started school after the previous school year was interrupted due to COVID-19. These students might need unique instruction that can challenge them and help them develop. On the other hand, most students struggled with the earliest levels of the game, suggesting that they need more work on the skills covered in those levels. 

Using a person-centered approach for reporting can help teachers understand where each student struggles and perhaps group students that may benefit from similar instruction. A person-centered approach provides an accessible starting point; for educators interested in the more fine-grained information, individual students’ answers and response times are also available. Using this approach can make game-based assessment results easier to interpret and thus more likely to be used as intended. We hope that this work is a step toward that goal.