Absent answers, SDP Fellow Aaron Butler helps Kentucky understand its new accountability measures.
In 2017, the Kentucky Department of Education (KDE) took a significant step to combat chronic absenteeism by including it as a measure of school quality. One of 36 states to focus on chronic absenteeism as an indicator of school quality or student success, Kentucky identifies a student as chronically absent when they miss more than ten percent of the school year. However, Kentucky included a unique distinction in their definition, one that marked a difference between instructional time and non-instructional time. Full-time equivalent (FTE) minutes, which measured instructional time during the school day, became the measure recommended to Kentucky schools and pointed to an emphasis on the academic impact of absenteeism on the state’s students.
Like many education departments, the KDE was thinking critically about how to use attendance data for school accountability. Recently, researchers have been making the case for the importance of student attendance by linking it to a cascade of other academic results or issues. Yet when the department released a chronic absenteeism report allowing schools and districts to track their chronic absence rates using the new FTE measure, the report was met with trepidation. With many leaders observing FTE rates rise and fall irregularly from week to week, Kentucky educators questioned the stability and reliability of its use in school accountability. The department took these concerns seriously and enlisted SDP Fellow Aaron Butler to investigate before any school was held to account.
“Imagine you’re a principal,” explains Butler, who was charged with examining the reliability of Kentucky’s new measure, “who’s trying to predict their school’s chronic absentee rates and gain points toward school accountability. You’re going to look back at prior years’ data. And if those data do not allow you to predict future rates with a reasonable amount of accuracy, you’re not going to put a lot of effort into that measure. Instead, you’re going to focus on the areas that bolster your school’s accountability rating the most.”
This was precisely the issue revealed through Butler’s project. Guided by the research question “How variable are school chronic absenteeism rates over time?”, Butler entered six years of raw student attendance data into a three-step data analysis process. First, he identified the chronically absent students—those enrolled in school for ten or more days who had an attendance rate of 90% or less, in minutes—for each school year. Next, he calculated school and district chronic absenteeism rates using this data on chronically absent students. Last, he fit a model to estimate schools’ 2016-17 rates from their 2015-16 rates. The results of this process indicated a considerable amount of variation in rates over time. Minutes, based on the sheer number included in a school day, set the conditions for much greater variation than do full- or half-time measurements.
Even the inclusion of additional years’ rates did not improve the prediction of future rates, which lead the KDE to some important considerations. One goal of adopting this new measure was to help schools understand the extent to which their students are regularly missing learning opportunities, thus arming them with data and motivation to keep students in school. But if the measure isn't reliable, would schools trust that they have a problem with chronically absent students? Would they believe that they should be held accountable for student absence rates?
In light of this project, KDE leaders decided to pause and refine the measure to get it right. For now, chronic absenteeism is still being reported on school report cards but is not included in school accountability.
Butler’s project deepens the conversation around data-driven decision making in education by telling a story about the importance of using the right data when holding schools to account. Tremendous efforts have been made to incorporate the recent deluge of education data into policy, district, and school decisions; and using attendance data for accountability is brand new. “I think people at all levels are working with data very smartly,” Butler reflects. “People are looking at prior years’ data to identify trends and glean insights.” Now, leaders are charged with more intensive tasks like modeling and testing new measures for validity and exploring potential unintended consequences.
Takeaways from Kentucky
Addressing chronic absenteeism is a laudable mission—everyone can get behind keeping kids in school, but using data in an evaluative manner puts people on high alert. It’s the state’s duty to hold systems accountable to doing what’s right for students and also to be thoughtful before doing so. In a climate of constant education policy churn, Kentucky took the bold steps of figuring out out how to move ahead confidently and fairly. The work Butler did in Kentucky carries important lessons for states choosing to use chronic absenteeism as an indicator of school quality in their accountability systems, including the 11 states who have listed enrollment cuts into their ESSA plans.
#1 - Definitions matter.
Kentucky’s decision to define enrolled instructional time in terms of FTE minutes led to a dramatically different story about chronic absenteeism rates in the state. Additionally, the department included a 10-day enrollment cut into its definition for inclusion in the measure along with other enrollment cuts, meaning that only students who were enrolled in the school for ten days of the school year or more were counted. These definitions all have possible unintended consequences that should (and are now) being explored before moving to implement the measure statewide.
#2 - Test and validate protocols.
KDE leadership recognized the opportunity to test the newly developed protocols used to process student attendance data for chronic absenteeism. Internal research projects, such as this project, create an opportunity to test the validity protocols and identify potential problems before the new measure is integrated into the accountability system. Additionally, by thoroughly examining these protocols in advance, agencies are better poised to create a strong case in favor of the measure to expand stakeholder buy-in from the beginning.
Aaron Butler is a Research Analyst for the Office of the Commissioner at the Kentucky Department of Education.