Reasons for Skepticism in K-12 Student Learning Report

A sweeping new review of national test data suggests the pandemic-driven jump to online learning has had little impact on children’s reading growth and has only somewhat slowed gains in math. That positive news comes from the testing nonprofit NWEA and covers nearly 4.4 million U.S. students in grades three through eight. But the report also includes a worrying caveat: Many of the nation’s most vulnerable students are missing from the data.

As educators work hard to support students in this challenging time, student achievement data are critical to needs assessment and instructional planning. Teachers need to know their students’ academic skills to tailor instruction, and policymakers need data to plan programs and interventions to catch up the students who have fallen behind. However, fall 2020 assessment results, administered in person or remotely, may not be capturing a significant portion of the student population. Many schools are not administering assessments at all due to technological and other challenges. Within schools that are testing, individual students are absent from school and/or opting out of testing for economic, health, technological, or other reasons unknown to educators and researchers.

Missing assessment data can lead to erroneous conclusions and decisions when students who are not assessed are systematically different from students who are assessed. COVID-19 learning losses and achievement gaps estimated from the data with such missingness will not reflect the larger student population. The most concerning scenario is that students not testing in fall 2020 are disproportionately from disadvantaged backgrounds. Not accounting for these students would produce underestimated learning loss and achievement gaps, potentially resulting in under-provision of support and services to the neediest students. Our supplemental attrition study viii used MAP Growth assessment data of nearly 5.2 million students who attended any grade between kindergarten and seventh grade in fall 2019 to

examine the patterns of missing data in fall 2020.

Online learning is causing a serious drop in academic achievement in Virginia’s largest school system, according to a Fairfax County Public Schools study, and the most vulnerable students — those with disabilities and English-language learners — are struggling the most.

Between the last academic year and this one, which for most students is taking place remotely, the percentage of F’s earned by middle school and high school students jumped from 6 percent of all grades to 11 percent — representing an overall increase of 83 percent from 2019 to 2020. Younger students were more seriously affected than older ones: Middle-schoolers reported an overall 300 percent increase in F’s, while high-schoolers reported a 50 percent increase. [snip]

“The pattern was pervasive across all student groups, grade levels, and content areas,” says the Fairfax study, published online this week. “The trend of more failing marks is concerning across the board but is especially concerning for the groups that showed the biggest unpredicted increases … namely our English learner students and students with disabilities.”

Learning losses in reading and math were determined by comparing annual growth from fall 2019 to fall 2020 to the average annual growth rate in prior years. Using data from a national sample of students who completed FastBridge’s adaptive reading and math assessments, results revealed modest reading losses across Grades K-8, modest math losses in early elementary grades, and substantial math losses across Grades 4–8.

These losses are generally in line with earlier predictions (Bielinski et al., 2020) that COVID-19 school disruptions would result in lower average fall screening scores.