Data, Data, Everywhere. Why So Hard to Use?
Good Data Management Practices: 3 Key Lessons
We live in a time that is organized around data in many areas of our lives. How these data are organized, analyzed, reported, and interpreted affects the utility of modern information systems. And that, in turn, affects how well these systems can support important school improvement planning and activities.
It’s easy to say that thoughtfully designed data systems are critical in our information economy. We’re thinking in particular about education systems that depend on clear, easily accessed, timely, and relevant data to make critical decisions about how to support students and where to deploy school resources.
But we also recognize that such systems do not come about easily. Designing, building, and managing data systems that 1) connect important data across systems, 2) ensure high data quality, and 3) support agile, timely, and accurate data use is much easier said than done.
Given this reality, we convened a session at the 2023 National Conference on Student Assessment to share some best practices in data design, reporting, and management in K-12 learning, assessment, and accountability.
Our three invited speakers—Ajit Gopalakrishnan from the Connecticut Department of Education, Mike Sao Pedro from Inq-ITS, and Dean Serrentino from Curriculum Associates—discussed their powerful work to create these integrated systems. In this blog post, we’ll outline their important insights and share the resources we presented (see links at the end of this blog post).
Message 1: Well-Managed Data Facilitates Better Data Use
If we want data to help us answer important questions about education, we need to have confidence in the quality of the data, and in the systems we use to store, analyze and report it. Our speakers consistently noted that integrated and user-friendly data systems—systems that bring together formerly disparate data sources and that generate useful tables and data visualizations, for instance—have empowered staff in their organizations, and school and district users, to do more with data.
A district leader who wants to make a data-informed decision about where to deploy reading specialists, for example, can do so more accurately and confidently using a single dashboard that displays academic reading results broken down in various ways, such as classroom, grade level, and school.
The key point here is to consider the needs of users: What data and resources will best support their access and use of critical data, and in what format? This also means attending to users’ ability to engage with data on different devices and with different approaches for their own sense-making purposes. We must do away with siloed, fragmented data systems because we want school and district staff to incorporate data use into their improvement practices.
Our presenters focused on key leverage points to move data-handling away from the all-too-typical “Jim’s got the file” scenario.
Message 2: Sustained Solutions Require Planning and Resources
Wouldn’t it be nice if we all could have fast, great, and cheap solutions? We know, though, that work to improve data architecture, engineering, and governance is a long-term investment with numerous benefits. It also requires consistent thought leadership and careful planning that engages all potential user groups. As we know from work on scaling and sustaining innovations, this kind of work is only achievable if it includes consideration of both its socio-cultural and technological dimensions.
Taking those dimensions into account means thinking through workflows, both internally and externally, and making hard decisions about what will and won’t work. This might include “freezing” data at particular points in time, for example, requiring all data to be stored centrally and processed in a particular manner, and letting go of some types of data if there is no clear justification for storing them.
Working with data, of course, isn’t just about static access to displays and information that are pre-processed in certain ways. It’s about having access to interactive tools that allow people to download and query segments or slices of the data in ways that evolve along with their understanding of the underlying phenomena—an interactive cycle.
Message 3: Data Champions Are as Important as Technical Solutions
Creating coherent data architecture, engineering, and analysis systems is more than a technical issue. A key challenge is to create a sustained data culture in which staff and data users share a mindset, practices, and tools oriented toward the same longer-term sense- and decision-making goals. They understand their roles in relation to data collection, analysis, and reporting; they are empowered to make connections across teams to increase data accuracy, integration, and/or usefulness, and they have systems and structures that focus on and support data integration and reporting efforts.
One concept that has gained increasing popularity in recent years is the “data champion,” a staff member whose job is perhaps more psychological or cultural than it is methodological or technical. Organizations have created role descriptions that specify key actions for the data champion, such as active listening across teams, continual advocacy for all things data, and strategic consultation across an organization’s layers. Education programs have been created for data champions (fittingly, exactly the kind of modern complex competencies we want our students to develop in K-12 settings). Our speakers all highlighted the importance of organizational leadership that understands and supports data integration efforts.
What kind of data management work are you doing in your organization? What are the challenges that you have overcome, and what strategies can you share with others? We would love to hear from you! Contact us at lpinsonneault@nciea.org and arupp@nciea.org to share learnings and continue the conversation!
Resources to support strategic and coherent data systems: