Can a teacher’s worth be measured by how much his students’ test scores improve? And should teachers who don’t move that needle very much be fired? These are two of the most controversial questions in education.
Some school districts have plunged ahead with “yes” answers to both. Whether that strategy — which leads to high teacher turnover — has improved student learning has yet to be fully analyzed. But a first stab at it has just been released by a quartet of researchers from the University of Virginia and Stanford University.
First, some background. It might seem pretty obvious that firing ineffective teachers and replacing them with better ones would be good for students. But identifying which teachers are ineffective is tricky. And it’s even trickier to dismiss unionized employees. Whether to pursue a dismissal has often been left up to the whims and energy of school principals. That’s why some policymakers came up with the idea of finding objective measures, and then adopting new firing policies.
Those who oppose measuring a teacher’s worth by test scores argue that the multiple-choice tests don’t capture a lot of what students are learning, and can underestimate great teaching. Even those who agree that the tests are somewhat good at identifying the worst teachers find other problems. For example, some teachers react by dumping creative lessons and teaching only topics that are on the standardized test.
And then there is the collateral damage from firings. It’s not just the teachers who are let go who suffer; it’s also disruptive to the entire school, possibly harming students. All the firings and new hirings devour principals’ energy and take time away from curriculum improvement and teacher training. And it can take months, even years, for new teachers to adjust to a new school. Previous researchers have found that schools with higher teacher turnover have lower student achievement. Even more troubling, a system that ranks teachers by test scores can stoke fear and resentment in the heart of any teacher. The stress might cause a school’s very best teachers to leave. Morale can plummet.
The academics from the University of Virginia and Stanford University set out to study how a system that increases turnover by firing low-rated teachers affects student learning. Specifically, they looked at the Washington, D.C., public school system from 2011 through 2013, after a new ratings-and-firing policy had been put in place there. The authors documented an 18 percent teacher turnover rate, one of the highest teacher departure rates among big cities in the country.
Despite the high turnover, the authors found that student performance, especially in math, improved. Roughly 400 good teachers left their jobs — about 13 percent of those rated “effective” or “highly effective”. But more than three times that rate, or 46 percent, of low performers left. That amounted to 130 teachers who were either automatically fired, because their ratings were so low, or quit of their own accord.
“They were replacing them with teachers who appear to be substantially better,” said Thomas Dee, an education professor at Stanford University, and one of four co-authors of the working paper, “Teacher Turnover, Teacher Quality and Student Achievement in DCPS,” published on the website of the National Bureau of Economic Research in January 2016. “Rigorous evaluation of teachers and then trying to replace consistently low-performing teachers appears to be effective.”
In the study, Dee and his research colleagues looked at the test scores of every grade from fourth through eighth in each school. If a fourth-grade teacher was rated low-performing and removed, they compared the test scores of fourth graders before and after the removal in that same school. The same was done for all departing teachers. After a high-performing teacher left, student performance usually went down. But after a low-performing teacher left, student performance usually went up.
When you net out these two effects, there was an overall positive gain in math scores, although one that might seem rather small (“8 percent of one standard deviation”). But Dee described it as equivalent to 10 percent of the black-white achievement gap, as measured by national tests. Reading scores rose less, and were only “weakly” significant from a statistical perspective, meaning that it’s unclear whether reading outcomes improved or not after the firing policy.
At the time of this study, a teacher’s rating was composed of several parts. Growth in student test scores accounted for 50 percent. The remainder was based on subjective teacher observations, principals’ ratings and the entire school’s test score improvements.
One critic of using student test scores to rate teachers said the study was “interesting” but not persuasive.
“D.C. was putting in place a lot of reforms during this period. Curriculum. Coaching. There are a lot of things that could account for the gain in scores,” said Linda Darling-Hammond, also a professor of education at Stanford, but not involved with this study. “When you see a change in scores, you have to have a way to evaluate all the policies; you can’t just say that the policy you’re interested in accounted for it.”
Darling-Hammond’s research has shown that rating teachers by student test scores, known as value-added measures, is flawed and volatile. Often a teacher with a high score one year will get a low score the next, and vice versa. Teachers of gifted students can find it hard to show test score growth because their students’ test scores start so high. Similarly, teachers of special-needs students or English language learners can have students who are so far behind that even if they catch up two grade levels, the test doesn’t capture that.
Still, it’s possible that the D.C. district was correctly identifying weak teachers and replacing them with better ones, and that their superior teaching skills were the reason student test scores improved. But it’s unclear if other school districts could replicate this, or even if D.C. could replicate this now. A big factor is the teacher job market. This study took place after the recession, when there were a lot of teachers looking for jobs. The city had the luxury of choosing from a great pool of applicants. With the economy’s rebound, there are now teacher shortages. And in other regions of the country, it might be hard to recruit good teachers altogether.
This article also appeared here.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.