How Federal Pandemic Aid Impacted Schools

K–12 schools received nearly $190 billion in federal relief during the COVID-19 pandemic, 90% of which went directly to local districts. Financially disadvantaged districts received the most aid money, but how effective was the money at helping students make up the learning they missed during the pandemic?

Answers can be found in new research which measured the impact of the spending by looking at the average test scores in reading and math from the spring of 2022–2023, for students in grades 3–8. The researchers were not able to assess which intervention strategies were the most effective because school districts were not required to report how they spent the funds they received.

Professor Thomas Kane, economist and co-author of the new report from the Center for Education Policy Research at Harvard University and The Educational Opportunity Project at Stanford University, explains the role that federal relief money played in the academic recovery story in 29 states.

Could you summarize what you found out about the impact of the federal relief money on student achievement during the 2022 to 2023 school year?

We found that $1,000 of federal aid per student that a district spent during the 2022–2023 school year was associated with a 0.03 grade equivalent rise in math achievement (or approximately 6 days of learning) and in reading, the effects were somewhat smaller, a 0.018 grade equivalent (or approximately 3 days of learning). So, the effects were not huge. I think readers might look at that and say, oh gosh, that's a small effect. But what people don't realize is just how strongly related to longer-term outcomes test scores are. So, although the impacts per dollar spent were not large, given the relationship between K–12 test scores and earnings later in life, our estimates imply they were large enough to justify the investment. 

In the conclusion of your report, you say that the average recovery was actually larger than what you expected based on your estimate of the effect of the spending. Why was that? 

We were surprised when we first got the 2022–2023 data and saw the total magnitude of the gains that year. They were 170% as large as the average annual improvement during the last period of rapid growth in achievement, between 1990 and 2013, in math and double the improvement in reading during that time period.

In this report, we investigated the role that the federal aid played in that growth. Our primary challenge was sorting out how much of the growth was due to spending, versus how much of the growth was related to community poverty — since poorer districts received more aid on average. We took several different approaches to doing that — for instance, using state differences in the Title I formula on which the funding was based and finding high-poverty districts which received large grants (because of the state they were in or because of anomalies in the aid formula) and similarly high-poverty districts with much smaller grants but similar prior trends in achievement. We tried multiple approaches and found similar answers each way we looked at it.

We're still surprised, partially because of the news over the past few years of districts spending the federal relief on athletics fields and across-the-board pay raises and the implementation challenges districts faced when trying to implement tutoring or recruiting students to summer school. But the dollars seem to have had an impact.

In your report you suggested that parental help at home, efforts on the part of teachers and students, and possibly increases in spending at the local level may have played a role in the recovery effort. And it's interesting because I remember the last time I talked with you, you mentioned your concerns about the lack of coordination with the spending of the federal relief money. Is that still a concern? 

Yes, in some ways the federal aid was like the first stage of a rocket — it got us started but was broadly focused and ultimately insufficient to get us all the way there. Part of that was due to a lack of coordination. Each district was developing and implementing plans largely on their own. It could have been much more effectively spent. For instance, research suggests that the cost effectiveness ratio for a high-dosage tutoring program was roughly 10 times as large as the cost effectiveness we found for each $1000 in aid spent.  

In the report, we also recommend efforts states should be doing now to continue the recovery, because it's pretty clear that there won't be another federal package, given what's happening in Washington. It’s alarming, but it’s just not on the radar screen of most governors — including here in Massachusetts, where the highest-poverty districts have actually lost additional ground since the pandemic. States have spent the last few years watching districts spend down their federal pandemic relief dollars, not recognizing that the recovery will not be complete when the federal dollars run out. Simply going back to business as usual will leave a lot of our neediest communities further behind than they were before the pandemic. So, we're hoping that these results become a call to action at the state and local level. It’s in governors and state legislators’ hands now. If they don’t step up, poor children will end up bearing the most inequitable and longest lasting burden from the pandemic. 

The aid did, by our estimates, seem to have a disproportionate effect on high-poverty districts, mostly because they got a lot more money. But that wasn't enough to completely offset the losses. The highest-poverty districts remain behind as well as the middle-income districts. The wealthiest districts we anticipate will be back to 2019 levels soon, not because they received much federal aid — they did not — but because they did not fall very far behind in the first place.

Are there lessons to be learned overall from the pandemic recovery effort? 

I do think it would have been beneficial to give federal regulators and state governments more opportunities to coordinate local efforts — like to plan statewide tutoring programs or to plan statewide summer learning programs. Most of the bigger districts would have had the staff to plan their own efforts, but the medium and smaller districts, they didn't necessarily have the bandwidth to be thinking about planning for major summer learning initiatives and tutoring programs. I think granting states, and the federal government, more say in approving local recovery plans, in ensuring that what districts were planning were sufficient to help students catch up and giving states more money to coordinate efforts would have helped. 

Imagine if, at the beginning of the pandemic, the federal government did not even try to coordinate efforts to develop a vaccine. Instead, suppose they took all that money and sent it to local public health departments, saying, “You figure it out.” Some would have succeeded, but many would have failed. That’s exactly what happened in the K–12 response. 90% of the federal aid went directly to local school districts. Some figured it out, but many did not.

States and districts should have plans on the shelf for what happens in the next pandemic. I'm sure there are individual schools that will say that they know exactly what they would do next time. But there has not been that sort of learning at the state level — since most states just took a back seat. I have not heard much planning at the state or federal level about what they would do differently next time — and how they might plan for a major tutoring initiative or assembling materials for summer learning, etc. We're not going to have better coordination next time unless somebody starts planning now.