The right to know what? - The Hechinger Report

Each fall, thousands of runners descend on the Big Apple to run the New York City marathon. They’ve trained hard all year, and give their all on the course. Long after the elite runners have finished, they stream across the finish line in clumps, exhausted at the end of their 26.2-mile journey. In the middle of the pack, as many as eight or 10 runners might cross the finish line in a single second, and nearly 400 in a single minute.

The difference between a time of 4:08:00 and 4:09:00, however, isn’t large enough to be important. It’s the difference between a rate of 9:28 per mile and 9:30 per mile. Given the vagaries of marathon running—the wind, the temperature, the features of the course—it would be unwise to conclude that the runner who crossed the finish line in 4:08:00 is a much better marathoner than the one who finished in 4:09:00.

But the runner with a time of 4:08:00 finished several hundred places ahead of the runner who finished in 4:09:00—surely that counts for something! Not really, I’d say. We can quantify the difference, both in absolute terms and in relative position, but these differences are not large enough to be meaningful.

The same is true of the information in the Teacher Data Reports recently released in New York City. Small differences in the estimated effects of teachers on their students’ achievement can appear to be much larger, because most teachers are about equally successful with the assortment of students they teach in a given year, regardless of whether those students begin the year as low-achievers or high-achievers. A trivial difference can appear much larger than it actually is, because, like the marathoners, many teachers are “crossing the finish line” at about the same time.

Since you made it to the bottom of this article, we have a small favor to ask.