Many education experts, and lay people alike, argue that you must set high academic standards in order for students to excel at school. All too often you hear the lament that low-income minority students often perform poorly because schools don’t expect much of them. Naturally, data geeks want to quantify abstract notions like “expectations,” and see exactly what they are. One way to do this is to look at where each state sets the passing score, or proficiency mark, on the exams it gives each year.
A branch of the U.S. federal government has actually gone through this exercise five times since 2003, and consistently finds that both math and reading expectations vary wildly throughout the country. As one analyst said recently, the eighth-grade proficiency level in one state might be equivalent to the fourth- or fifth-grade proficiency level in another state.
But these federal analyses are almost impenetrable to non-statisticians. The differences are measured in “NAEP points” and “standard deviations.” And you’re left wondering if these differences are significant, or merely interesting to academics who analyze measurement errors.
So I thought I would take the most recent report from the National Center for Education Statistics (NCES), called “Mapping State Proficiency Standards Onto NAEP Scales: Results from the 2013 NAEP Reading and Mathematics Assessments,” and released July 9, 2015, and convert it into something both my mother and my daughter can understand: grade levels, as in the difference between sixth and seventh grade. I found that 26 states set expectations that were three or more grade levels behind the eighth-grade standards of New York State, the state that had set the highest expectations back in 2013, as an early adopter of Common Core.
Eighth-grade reading expectations, as set by each state’s annual test, can be several grade levels apart.
Zoom in and click on any state to see what each state’s proficiency mark on its 8th-grade reading test is equivalent to on the national NAEP exam, and how many grade levels that is below New York, the top state. (Interactive map created by Jill Barshay of The Hechinger Report. Source data: NAEP mapping report, 2013 NAEP scores, Jill Barshay’s calculations.)
Eighth-grade math expectations, as set by each state, also vary wildly.
Zoom in and click on any state to see what each state’s proficiency mark on its 8th-grade math test is equivalent to on the national NAEP exam, and how many grade levels that is below New York, the top state. California and Virginia didn’t assess general mathematics in 8th grade in 2013. (Interactive map created by Jill Barshay of The Hechinger Report. Source data: NAEP mapping report, 2013 NAEP scores, Jill Barshay’s calculations.)
This number-crunching exercise was inspired by comments from Gary Phillips, a former NCES acting commissioner, and now a vice president at the American Institutes for Research. In an online press briefing just prior to the release of the July report, Phillips said that “states are setting wildly different standards and most states are setting very low standards.” To make it more vivid, Phillips explained that differences between the states with highest expectations and those with the lowest were equivalent to “three or four grade levels.”
It’s worth emphasizing that these are not measurements of how kids are actually doing. New York’s students, on average, aren’t high-performing at all. Eighth graders nearly 30 states scored better than this high-expectations state in 2013. This report measures where each state sets its own proficiency mark on its own state test. Some tests are much easier than others. And some states set a proficiency score much higher than others. So in order to compare state tests against each other, the NCES mapped 50 different tests and their 50 different proficiency scores onto a single ruler, a national test known as the National Assessment for Education Progress or NAEP.
States typically test students annually, but NAEP is administered for only fourth, eighth and 12th grades and only every other year. The NCES mapping study was further limited to fourth and eighth grade only.
In eighth grade reading, for example, there are 83 NAEP points between where New York, the top state, set its proficiency mark (at 282), and where Georgia, the lowest state, set its proficiency mark (at 199).
How much is 83 NAEP points?
Fortunately, the fourth grade NAEP test uses the same scale as the eighth grade NAEP test, so we can compare them. The average fourth grader in the United States scored 222 on the NAEP reading test, while the average eighth grader scored 46 points more, or 268. That means the 83-point gap in expectations was substantially larger than the 46-point gap between what the average fourth and eighth graders can do! That’s pretty astounding.
As an exercise, I divided the 46 NAEP points by the four grades between fourth and eighth, and then attributed 11.5 NAEP points to each grade level. That allowed me to calculate how many grade levels each state lagged behind New York for the interactive maps that are part of this story.
I went through a similar process for math, where there was a 60-point gap in 8th grade expectations between the top state — again New York — and the bottom state — this time Connecticut (which happens to be a very high-performing state, despite low math expectations). And I found that 10.75 NAEP points was roughly equivalent to a grade level. So again, I found that the gap in expectations for a single grade level, such as eighth grade, is larger than gap between the actual performances of the average fourth and eighth grader.
For both math and reading, I capped the expectations gap at four grade levels. That is, even if my number crunching concluded that a state’s proficiency standard was seven grade levels behind New York’s, I lumped it into the “four or more” category, colored in dark brick-red in the maps.
To be sure, NCES doesn’t approve the use of its NAEP scales for calculating grade levels. They weren’t constructed for that purpose. But as a number-crunching journalist, I can do that. And the reason why I took the time to do this is that I don’t think most Americans — or even most education experts — are aware of just how big the expectation gaps are in our country.
Another big caveat is that this data is from 2013. Many states have since raised their goals for students and instituted more difficult tests. Among the 44 states which have adopted Common Core, it will be interesting to see if any of them buttress these higher standards with a higher proficiency mark on annual tests.* And we’ll have to wait until NCES’s next mapping exercise to see how it all shakes out.
*Correction: This column has been updated to remove a sentence stating that states who use nationally produced Common Core-aligned tests could set their own cut scores.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.