Here’s a riddle. In New York City, the largest school district in the nation, eighth-grade math scores were virtually unchanged between 2013 and 2015, according to national test results released October 28, 2015. That’s not great news, but at least they didn’t decline along with those of the rest of the nation’s eighth graders. (See our story about the 2015 National Assessment of Educational Progress, or NAEP, here).
But a closer inspection of New York City’s scores showed something peculiar. Scores of white students fell a whopping seven points, with the average white student now scoring below “proficient” instead of above. Scores of black students declined two points. And Asian test scores declined by a point as well.
So, three of the city’s four major demographic groups, which make up more that 60 percent of the city’s public school children, did worse. But overall the city’s average math score was unchanged. How can this be?
It’s partly explained by the four-point rise in the eighth-grade math scores of Hispanics, who make up 37 percent of the population. But the bigger answer is that U.S. cities are fluid and their populations can change fast.
In 2013, New York City’s eighth graders were 11 percent white, 31 percent black, 42 percent Hispanic and 16 percent Asian. Two years later, in 2015, the city’s gentrification had changed the complexion of middle school classrooms. The city’s eighth grade was suddenly 15 percent white, 30 percent black, 37 percent Hispanic and 17 percent Asian. The two demographic groups that, on average, score low — blacks and Hispanics — had shrunk (down 6 percentage points). The two demographic groups that score high — whites and Asians — had grown (up 5 percentage points). Even with the sharp decline in white scores, white students, on average, were still scoring much higher, about 30 points higher, than their black and Latino peers. And so when there are more whites and Asians in the public school system, that’s going to prop up the school district’s overall scores.
Philadelphia provides an even starker example of how urban gentrification can mask academic declines. There, white, black and Hispanic scores in eighth-grade math all declined between 2013 and 2015, but the Department of Education reported that, overall, the city’s eighth-grade math score was unchanged. That’s because whites had jumped to 19 percent of the eighth grade population from 16 percent. And Asians, whose scores did rise, rose to 9 percent from 7 percent. Meanwhile, the low-scoring black population fell by five percentage points to 51 percent of Philadelphia’s eighth grade classrooms. (The Hispanic population rose slightly from 18 to 19 percent).
The statistical phenomenon is known as Simpson’s Paradox, when subgroups move in one direction, but the whole moves in another. Another type of Simpson’s Paradox has been seen in the national NAEP scores for years. Over the long term, since the early 1980s, each racial or ethnic group has shown substantially more growth on NAEP assessments than the nation has as a whole. That’s because the proportion of white students has steadily fallen and the proportion of minorities has steadily increased. Statisticians believe the U.S. public school system flipped from majority white to majority “minority” in 2014, something we’ll have actual confirmation of in a couple years when demographic data are finalized.
Could the increase in minorities and in childhood poverty nationwide explain the terrible deterioration of national test scores in the past two years? I put this question to Matthew Chingos of the Urban Institute, who recently adjusted NAEP scores throughout the country to account for racial and ethnic, poverty and language differences in each state. He calculated that changing demographics had indeed caused progress to be understated — up through 2013. But now, he told me, the sharp 2015 score declines cannot be explained by increases in the proportion of disadvantaged students.
“If you just look at the size of the score change from 2013 to 2015, it seems way too large to be explained by demographics, particularly in eighth grade, where the drop was the largest,” said Chingos.
I looked at the eighth-grade math scores of white students who are not poor enough to qualify for the federal lunch program. Those non-poor white scores declined in most states across the country from 2013 to 2015. Even if demographics partly explains the national drop, there were real declines among children who are not disadvantaged, as well.
But Chingos says cities are different. “Cities can change more quickly than a state can,” he said. “The demographics of a place like D.C. can shift much more quickly than even a small state, such as Nevada or Rhode Island. It’s even more important to be thinking about demographics for cities. It’s not just about shifting demographics of people who live in the city, but shifting demographics of people who send their kids to the public schools.”
Simpson’s Paradox didn’t manifest itself in every city. I looked across the 22 large cities that participate in extra testing designed to give the public more data on their students. And I found that the white population had increased in seven of these cities in the two- year period from 2013 to 2015. They include Washington, D.C, Miami and Atlanta. The black population had fallen in 15 cities. (Demographic data was not available for two of the 22 districts.)
It’s not clear urban school leaders are yet aware that many of their stable NAEP scores are due to demographic shifts. But New York City is concerned enough that it is planning to double down on math for all demographic groups, especially because its fourth-grade math scores dropped sharply citywide. In an e-mailed response to my questions, City Hall spokesman Wiley Norvell wrote that school leaders are working on fixing the curriculum and “training teachers to deliver it.”
This article also appeared here.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.