When Candice McQueen learned last fall that a controversial statistical analysis had declared her teacher-training program relatively weak in the area of social studies, she wasn’t surprised.
Earlier feedback, including postgraduation surveys, had suggested that the college of education at Nashville’s Lipscomb University needed to bolster its social studies training, said McQueen, the college’s dean. But the state data spurred McQueen to act more quickly. College faculty and administrators began scrutinizing the social studies curriculum and training approach, partly in the hope of warding off future embarrassment.
Scores of teacher-training programs across the country will likely face similar scrutiny in the coming years. Following the lead of Tennessee and Louisiana, policymakers in a growing number of states are evaluating the programs — and even issuing report cards for them — based on the test scores of their graduates’ students. So far, eight states have policies requiring them to do a similar analysis, most of them adopted in the last few years, according to the National Council on Teacher Quality (NCTQ), a research and advocacy group that supports higher standards for schools of education.
“This is a policy movement that’s sweeping the country,” said Charles Peck, a professor of special education and teacher education in the University of Washington’s College of Education.
Related efforts to evaluate individual teachers based on student test scores have sparked a flurry of publicity — and led to a federal lawsuit filed by a group of Florida teachers who complained they would be rated on the test scores of students who weren’t even in their classes. But those efforts targeted at preparation programs (which include long-standing university-based schools of education and less-traditional programs like Teach For America) have gone comparatively unnoticed and unexamined.
As other states follow a similar path, the experience in Louisiana and Tennessee speaks to the promise and peril of the new approach. Some programs, like McQueen’s, have used the data to make improvements. “I’m a big believer in never looking at just one piece of program data,” said McQueen. “But this encouraged us to move faster than we might have.”
Some worry, though, that the data can be overly simplistic, and misleading at times. Do low reading scores recorded years after a group of teachers enter the classroom, for instance, mean their training program had a bad curriculum or weak instructors? Or did it admit weaker candidates from the start, or perhaps send them off to schools with less supportive principals?
“It’s kind of like having a fire alarm go off in your house, but not knowing where the fire is,” says Peck.
Pinpointing weaknesses
The Tennessee Higher Education Commission reports have tied student performance back to their teachers’ preparation programs since 2008. By using what’s known as a “value-added” analysis, researchers homed in on the amount of growth seen in individual students, no matter their starting point. They then analyzed the overall student growth in the classrooms of recent graduates of different training programs.
The Memphis Teacher Residency program and Teach For America-Memphis earned the highest marks of any Memphis-area training programs when the 2012 Tennessee Report Card on the Effectiveness of Teacher Training Programs was released last fall. Graduates of those two programs significantly outperformed University of Memphis graduates, whose students on average performed in the bottom 20 percent in reading in grades 4-8; the for-profit Victory University in Memphis; and TNTP (formerly The New Teacher Project) Memphis Teaching Fellows.
E. Sutton Flynt, head of teacher education at the U of M, told The Commercial Appeal last October that the university’s results will improve in the next round when curriculum and other changes are reflected in the data. He also criticized the state for looking at only “one piece of a panoramic photo and extrapolating the results.”
“We have a 100-percent pass rate on what the state requires,” he said.
Katrina Miller, director of Tennessee’s federal First to the Top grant, said some of the state’s teacher-preparation programs were reluctant to have their results publicly reported. But most have come around over time; some, like Lipscomb, have even used the data to shore up their programs.
McQueen said Lipscomb’s review of the social studies program revealed that the history department, unlike others, did not have a strong advocate or liaison working directly with the education students. History majors also did not receive as much clinical experience as aspiring teachers who were majoring in other subjects. “We’ve made changes on both of those fronts, and we’re hoping those will pan out,” she said.
It’s not always so easy for an institution to pinpoint the source of a problem, however. For that reason, Tennessee officials designed a set of 10 research questions the programs can use to determine why they might be seeing a particular result. One question asks how many hours teacher candidates spent doing clinical training at schools, for instance.
George Noell, a Louisiana State University researcher who designed Louisiana’s new evaluation system, said he is pleased that some of the lowest-performing programs are making improvements, although “across the whole spectrum, it’s not as clear the data have helped people as much as I hoped.”
“I assumed it would not be as long a journey from seeing results to figuring out what to do with them,” he said. “But I consider the fact that we are even talking about it huge progress.”
Admissions standards and selectivity
Some experts say that determining how much of a young teacher’s success or failure can be tied to his or her training program is like asking which came first, the chicken or the egg.
If a program’s graduates post weak results, “It could be that they are doing a fabulous job, but they are not selective enough in terms of who is admitted,” said Kate Walsh, president of NCTQ. She pointed out that many of the programs that have performed best on Tennessee and Louisiana’s evaluations also have highly competitive admissions.
Tim Daly, president of the national organization TNTP, which has trained teaching fellows in Memphis, disagreed that the selectivity of a program significantly affects its graduates’ performance in the classroom. “While selectivity may play a small role, what we do to train teachers plays a big role,” he said.
Miller said she sees selectivity as something that training programs can control. “You are choosing who to admit,” she said. “Selectivity, since it’s chosen by the program, is part of the training.”
Some programs are far more oversubscribed than others, however. And it’s easier for Teach For America — where more than 10 percent of all Ivy League graduating seniors apply — to be highly selective than the average state school. That said, some teacher-training programs have raised their admissions standards partly as a result of the new reports. Lipscomb now requires students to have a 2.75 or 3.0 grade-point average (depending on the program), rather than a 2.5, to enter the education school.
What lies ahead
So far, there have been no sanctions for teacher-training programs that consistently post weak results in Tennessee. In Louisiana, the state can shut down persistently failing programs, although none has reached that point.
“I don’t view it so much as an accountability mechanism because none have been shut down,” said Daly.
Tennessee plans to start using the report cards for accountability purposes, but has yet to figure out the timeline and details, said Miller. “We’ve taken the first three or four years to make sure the data is accurate and know that the report card is a solid product,” she said.
While both Tennessee and Louisiana have been forerunners in using quantitative data to evaluate their teacher-training programs, neither has done much yet on the qualitative side. That will likely change this school year as the Tennessee report cards begin to reflect scores from the state’s new teacher evaluation system, which includes multiple classroom observations, said Miller.
Peck said it’s essential to incorporate a mixture of qualitative and quantitative elements when assessing the programs.
“We’re learning how important it is to have more than one measure of teacher effectiveness,” said Peck. “If you have multiple measures, it is much more powerfully predictive.”
How about non-test-related aspects of teaching? Are you not interested in them, since they are difficult to squeeze into your quantitative models? Did not the teachers who saved the children’s lives at Sandy Hook or in Moore, Oklahoma add value to their schools and communities? If so, how do you measure that value? How do those sacrifices and accomplishments fit into your formulas? When teachers form relationships with students that enable them to prevent those students from dropping out, and those students later become employed, including sometimes in schools, as teachers or in other capacities, do those virtues appear in your evaluations of the teachers?