Earlier feedback, including postgraduation surveys, had suggested that the college of education at Nashville’s Lipscomb University needed to bolster its social studies training, said McQueen, the college’s dean. But the state data spurred McQueen to act more quickly. College faculty and administrators began scrutinizing the social studies curriculum and training approach, partly in the hope of warding off future embarrassment.

Lester School assistant principal Dr. Isaac Robinson evaluates fourth grade teacher Debra Holt-Robinson’s (no relation) class at the school. Robinson observes the teacher and the lesson then moves through the class to see how well the students grasp the concepts taught. (Mike Brown/The Commercial Appeal)

Scores of teacher-training programs across the country will likely face similar scrutiny in the coming years. Following the lead of Tennessee and Louisiana, policymakers in a growing number of states are evaluating the programs — and even issuing report cards for them — based on the test scores of their graduates’ students. So far, eight states have policies requiring them to do a similar analysis, most of them adopted in the last few years, according to the National Council on Teacher Quality (NCTQ), a research and advocacy group that supports higher standards for schools of education.

“This is a policy movement that’s sweeping the country,” said Charles Peck, a professor of special education and teacher education in the University of Washington’s College of Education.

Related efforts to evaluate individual teachers based on student test scores have sparked a flurry of publicity — and led to a federal lawsuit filed by a group of Florida teachers who complained they would be rated on the test scores of students who weren’t even in their classes. But those efforts targeted at preparation programs (which include long-standing university-based schools of education and less-traditional programs like Teach For America) have gone comparatively unnoticed and unexamined.

As other states follow a similar path, the experience in Louisiana and Tennessee speaks to the promise and peril of the new approach. Some programs, like McQueen’s, have used the data to make improvements. “I’m a big believer in never looking at just one piece of program data,” said McQueen. “But this encouraged us to move faster than we might have.”

Some worry, though, that the data can be overly simplistic, and misleading at times. Do low reading scores recorded years after a group of teachers enter the classroom, for instance, mean their training program had a bad curriculum or weak instructors? Or did it admit weaker candidates from the start, or perhaps send them off to schools with less supportive principals?

“It’s kind of like having a fire alarm go off in your house, but not knowing where the fire is,” says Peck.

Pinpointing weaknesses

The Tennessee Higher Education Commission reports have tied student performance back to their teachers’ preparation programs since 2008. By using what’s known as a “value-added” analysis, researchers homed in on the amount of growth seen in individual students, no matter their starting point. They then analyzed the overall student growth in the classrooms of recent graduates of different training programs.

The Memphis Teacher Residency program and Teach For America-Memphis earned the highest marks of any Memphis-area training programs when the 2012 Tennessee Report Card on the Effectiveness of Teacher Training Programs was released last fall. Graduates of those two programs significantly outperformed University of Memphis graduates, whose students on average performed in the bottom 20 percent in reading in grades 4-8; the for-profit Victory University in Memphis; and TNTP (formerly The New Teacher Project) Memphis Teaching Fellows.

E. Sutton Flynt, head of teacher education at the U of M, told The Commercial Appeal last October that the university’s results will improve in the next round when curriculum and other changes are reflected in the data. He also criticized the state for looking at only “one piece of a panoramic photo and extrapolating the results.”

“We have a 100-percent pass rate on what the state requires,” he said.

Katrina Miller, director of Tennessee’s federal First to the Top grant, said some of the state’s teacher-preparation programs were reluctant to have their results publicly reported. But most have come around over time; some, like Lipscomb, have even used the data to shore up their programs.

McQueen said Lipscomb’s review of the social studies program revealed that the history department, unlike others, did not have a strong advocate or liaison working directly with the education students. History majors also did not receive as much clinical experience as aspiring teachers who were majoring in other subjects. “We’ve made changes on both of those fronts, and we’re hoping those will pan out,” she said.

It’s not always so easy for an institution to pinpoint the source of a problem, however. For that reason, Tennessee officials designed a set of 10 research questions the programs can use to determine why they might be seeing a particular result. One question asks how many hours teacher candidates spent doing clinical training at schools, for instance.

George Noell, a Louisiana State University researcher who designed Louisiana’s new evaluation system, said he is pleased that some of the lowest-performing programs are making improvements, although “across the whole spectrum, it’s not as clear the data have helped people as much as I hoped.”

“I assumed it would not be as long a journey from seeing results to figuring out what to do with them,” he said. “But I consider the fact that we are even talking about it huge progress.”

Admissions standards and selectivity

Some experts say that determining how much of a young teacher’s success or failure can be tied to his or her training program is like asking which came first, the chicken or the egg.

If a program’s graduates post weak results, “It could be that they are doing a fabulous job, but they are not selective enough in terms of who is admitted,” said Kate Walsh, president of NCTQ. She pointed out that many of the programs that have performed best on Tennessee and Louisiana’s evaluations also have highly competitive admissions.

Tim Daly, president of the national organization TNTP, which has trained teaching fellows in Memphis, disagreed that the selectivity of a program significantly affects its graduates’ performance in the classroom. “While selectivity may play a small role, what we do to train teachers plays a big role,” he said.

Miller said she sees selectivity as something that training programs can control. “You are choosing who to admit,” she said. “Selectivity, since it’s chosen by the program, is part of the training.”

Some programs are far more oversubscribed than others, however. And it’s easier for Teach For America — where more than 10 percent of all Ivy League graduating seniors apply — to be highly selective than the average state school. That said, some teacher-training programs have raised their admissions standards partly as a result of the new reports. Lipscomb now requires students to have a 2.75 or 3.0 grade-point average (depending on the program), rather than a 2.5, to enter the education school.

What lies ahead

So far, there have been no sanctions for teacher-training programs that consistently post weak results in Tennessee. In Louisiana, the state can shut down persistently failing programs, although none has reached that point.

“I don’t view it so much as an accountability mechanism because none have been shut down,” said Daly.

Tennessee plans to start using the report cards for accountability purposes, but has yet to figure out the timeline and details, said Miller. “We’ve taken the first three or four years to make sure the data is accurate and know that the report card is a solid product,” she said.

While both Tennessee and Louisiana have been forerunners in using quantitative data to evaluate their teacher-training programs, neither has done much yet on the qualitative side. That will likely change this school year as the Tennessee report cards begin to reflect scores from the state’s new teacher evaluation system, which includes multiple classroom observations, said Miller.

Peck said it’s essential to incorporate a mixture of qualitative and quantitative elements when assessing the programs.

“We’re learning how important it is to have more than one measure of teacher effectiveness,” said Peck. “If you have multiple measures, it is much more powerfully predictive.”

There’s a ton of education data out there now. Every state in the nation now maintains a longitudinal data system that tracks each student’s test scores year after year. (That’s thanks to more than $620 million in federal funds for setting up state data systems since 2005, plus additional Race-to-the-Top grants). But McCann and Kabaker make the argument that, for the most part, they’re not being used by teachers to figure out how to teach their students better.

McCann and Kabaker describe recent efforts in Oregon and Delaware to get teachers to actually use the data. In both cases, it was time consuming. One Oregon school district got the school board to approve a later start time to the school day so that teachers could pore over data in the morning. Other schools set aside regular blocks of time during the school day for teachers and administrators to meet. Delaware hired professional data coaches from Wireless Generation, a private company now owned by Rupert Murdoch’s Newscorp. The idea was to examine the data and plan instructional changes, such as when to use whole group versus small group or individual instruction. Or the teachers could identify which students need additional help. Some teachers were resentful that it was taking away from conventional lesson planning. Many teachers struggled to find the required hours. And McCann and Kabaker point out there’s only so much you can do with year-end test data.

What data freaks really want are so-called “formative” tests that children take many times through the year so that you can see how much they’re learning before the year is over when there’s still time to make adjustments. But formative tests are at their infancy and there’s a lot of push back against adding more tests to the school year. Oregon didn’t have any formative assessments, but it is now starting some pilots.

The results?

In Oregon, schools that participated in the data program tended to see their reading scores increase more than schools that didn’t participate. It’s important to note that the participating schools tended to be lower performing at the start, back in 2008. By 2012, the students of the data-analyzers had, on average, surpassed the reading scores of the non-analyzers, but just by a hair (80.52 vs. 79.62). It was not as rosy in math. Students of the data analyzers did improve and close the achievement gap. But the non-analyzers were still outperforming the analyzers at the end.

In Delaware there is no data on whether the data coach program is working. That’s because Delaware was experimenting with other reforms at the same time and it’s difficult to say how much of a role the data analysis alone had on student achievement. But the state department of education is working to produce an independent analysis.

Delaware benefited from Race to the Top grants to fund its data coach program, but McCann and Kabaker are worried that Congress will cease funding it in 2014. Even President Obama’s budget doesn’t include any Race to the Top money for elementary and high schools.

Instead, the authors point to the “Improving Teacher Quality State Grants” program for advocates of data-driven instruction. It is currently used for class-size reduction and teacher training programs. The authors want Congress, when it reauthorized the No Child Left Behind Act, to explicitly promote the use of data training projects with these funds instead. The authors also want the Department of Education to redesign the statewide longitudinal data systems grants to reward proposals that focus on the use of data, not just the existence of the data.

The spotlight on teacher evaluations widened last June when Los Angeles County Superior Court Judge James Chalfant ruled that the district was violating California’s longstanding teacher evaluation law, the Stull Act, by not ensuring test scores were used.

The 1971 law, signed by then-Gov. Ronald Reagan and named after a former Republican lawmaker, requires student achievement to be included in teacher evaluations – something Los Angeles Unified, and most districts, resisted for decades. Some districts found they didn’t need to fully comply with the Stull Act to receive millions of dollars from the state designated for linking teacher evaluations to student achievement.

Melrose Elementary teachers and its principal, Bernadette Lucas, took part in Los Angeles Unified School District’s pilot teacher evaluation program, which factored in student achievement to determine teacher effectiveness.
Credit: Carlos A. Moreno / The Center for Investigative Reporting

The law was amended in 1999 under Gov. Gray Davis, requiring school boards to evaluate teachers based on state test scores as they “reasonably relate” to a teacher’s classroom performance, a vague term that effectively made it easy for districts to avoid the law.

Now, the little-known Stull Act is having its day after more than 40 years of virtual neglect.

The recent case, Doe v. Deasy, was filed in 2010 by several students in the district and sponsored by EdVoice, the nonprofit education reform group backed by billionaire Eli Broad, Netflix CEO Reed Hastings and Richard Merkin, CEO of the Heritage Provider Network, among others.

Chalfant ordered the district to negotiate the terms of test score use with the teachers’ union. All told, the case and bargaining have cost the district more than $418,000 in legal bills to date, district officials said, not including $550,000 in attorney’s fees paid to the plaintiffs.

Los Angeles Unified officials were not displeased with the outcome. The ruling served as an endorsement of the district’s efforts to include student test scores in teacher evaluations. The local union, which has had an acrimonious relationship with the Los Angeles administration, also won something it had been seeking: a voice in the matter.

The judge “was actually causing to happen something we had wanted to have happen for a long time, which is that the teachers and the school board slash administration were in the same room talking about what a system would look like,” said Warren Fletcher, president of the 36,000-member United Teachers Los Angeles union.

The union had opposed the district’s effort to test changes to the system without negotiating the changes in teacher workload that came with it, and it unsuccessfully sought an injunction from the state Public Employment Relations Board to stop the program in 2011.

Following Chalfant’s ruling, the two parties spent months negotiating before reaching a tentative deal in November, ratified earlier this year. By next year, Los Angeles will comply with the intentions of the four-decade-old law.

The deal allows for state test scores and the Academic Growth Over Time measurement, which includes past test scores and demographics like students’ family income, language and ethnicity. But the deal specifically prohibits the use of individual teacher growth scores other than to “give perspective and to assist in reviewing the past CST (California Standards Test) results of the teacher.”

Critically, however, the agreement is silent on how much weight student data has. Los Angeles Unified Superintendent John Deasy has said it should account for up to 30 percent of a teacher’s evaluation; the union responded by saying such a “cookie-cutter” approach violates its agreement with the district.

Although a judge found Los Angeles Unified out of compliance with the Stull Act last summer, the district had received nearly $8.5 million in funds from the state for costs related to compliance with the law dating back to the 1997-98 school year, state records show.

Since 2004, school districts in California have been eligible to obtain certain reimbursements from the state controller’s office for enforcing the Stull Act dating back to 1997. Reimbursable costs include time spent reviewing a teacher’s California Standards Test scores and providing a written performance assessment based on the scores for legally required evaluations.

The flow of money has been unreliable. To date, the state controller’s office has received $236.3 million in Stull Act claims, but not enough funding has been appropriated by the Legislature each year to pay them. One-quarter of the claims for work through the 2010-11 school year have been reimbursed, state data shows.

But the claims from some districts have proved to be suspect.

Eleven years of Stull Act claims, totaling almost $1.3 million, were audited in the Oceanside Unified School District in 2011; $16,536 was found to be allowable. The controller disallowed eight years of staff time claims because the district refused to produce sufficient backup documentation.

Locating the documents now “would be a significant drain on district resources, staff and funds,” Oceanside district staff wrote.

The audit also noted that district was not using state test scores in teacher evaluations during the 11-year period reviewed and that the district’s contract with the teachers union at the time did not allow for it.

The Rowland Unified School District, roughly 30 miles east of Los Angeles, also came up short in its 2010 Stull Act claims audit, public records show. Out of the $2.9 million in reimbursements submitted in the nine years from 1997 to 2006, less than $683,000 was valid, the controller found, primarily due to a lack of supporting documentation.

Los Angeles Unified filed its first Stull Act claim in five years in February. The district is seeking $599,222 in reimbursements for principal and assistant principal staff time spent on evaluations last year, records show.

Alternative reform efforts

Although the Stull Act, and the money that has traditionally flowed from it, has been largely unsuccessful in pressuring California districts to adopt evaluations that measure teacher quality by the performance of their students, recent federal efforts have had more success.

The Obama administration has used federal money for the Teacher Incentive Fund, Race to the Top and School Improvement Grant program to encourage policymakers in states and districts to adopt new teacher evaluations that incorporate student test scores, despite the controversy surrounding them.

And now, the U.S. Department of Education has taken advantage of the threat of upcoming sanctions for schools and districts that don’t meet student proficiency targets under the federal No Child Left Behind law to extract agreements from states that they will overhaul their evaluations and attach them to student growth. Places that agree to make changes can receive waivers from the law’s requirements – and ensure that the federal funds upon which many depend, including Title I, will continue to flow. Thirty-seven states and the District of Columbia have had waivers approved. Nine other states are under review.

Los Angeles is one of nine districts in California applying for a waiver as a group; the waiver was amended last week to make it stronger and clearer, officials said.

It’s still unclear whether the revival of the Stull Act in Los Angeles or the rising federal pressure will lead to new evaluations in other California districts. But on a state level, efforts to change and clarify existing teacher evaluations laws and the use of state test scores have gained some traction in recent years.

AB 5, introduced in 2010 by then-Assemblyman Felipe Fuentes, a Sylmar Democrat, was withdrawn in August before a vote was taken, despite support from the California Teachers Association and California Federation of Teachers. The bill would have made the Stull Act ineffective in July 2014 in favor of local collectively bargained agreements.

Two Republican-backed measures introduced in February faced significant challenges in the Legislature, which is controlled by Democrats.

Republican Sen. Bob Huff of Brea, whose district includes part of Los Angeles County, introduced legislation that would have required districts to use a “rigorous, transparent, and fair multiple-measures evaluation system” for teachers based on research-proven measures of effectiveness by the 2016-17 school year. The bill also would have given districts the right to assign or transfer teachers based on effectiveness. Huff conceded defeat in late April after the Senate Education Committee failed to pass the bill.

A bill by Republican Assemblywoman Kristin Olsen of Modesto would take things even further sooner.

Olsen’s AB 430 would require districts to negotiate an evaluation system with their unions by Jan. 1, 2015, that counts value-added scores for at least one-third of an evaluation. If no such deal is reached, 50 percent of an evaluation must be based on pupil progress on a test given at the beginning and end of the school year. The other 50 percent would be based on class observations by the principal and peers. The system would begin in the 2015-16 school year.

“Teachers should have the ability to have organized and meaningful reviews so that professional growth is tangible, not an elusive and arbitrary concept,” Olsen said in a statement. In May, she canceled a scheduled Assembly Education Committee hearing, though it may be taken up at a later date.

Ashly McGlone produced this story for The Hechinger Report and The Center for Investigative Reporting.

“I am just an average teacher, according to that data, but if you look at my class and see how many children are proficient and advanced, it’s a lot of kids,” said Wynne Davis, now an instructional support coach for Melrose Elementary in central Los Angeles.

Seeing her assessment as an “average” teacher in English and a “more effective than average” teacher in math hit the Los Angeles Times a few years ago was upsetting and embarrassing, she said. The assessment of her teaching was based in part on her 2010 students’ test scores.

Sachiko Miyaji was one of five teachers in the pilot evaluation program last year at Melrose Elementary School. She said student test scores don’t measure the vibrancy of what’s happening in the classroom.
Credit: Carlos A. Moreno / The Center for Investigative Reporting

“I think we have to be judged by how well the kids do in our classrooms, because that’s the nature of teaching. What I don’t like is it doesn’t seem fair,” she said.

Nowhere else in California has the debate over the use of student test scores to grade teachers gained more attention than in the Los Angeles Unified School District. The second-largest school district in the nation at more than 640,000 students, Los Angeles Unified has become a testing ground to increase accountability for teachers, a movement that has gained speed across the nation.

The hope is that schools will improve student achievement by better identifying which teachers are excelling, which are struggling and which need to be removed from the classroom altogether.

The outcome in Los Angeles will have repercussions throughout the state, as pressure mounts to improve the state’s lagging achievement and qualify for federal funding.

The district ­– propelled by a lawsuit and led by Superintendent John Deasy, a darling of reformers who support the effort to overhaul teacher evaluations – has pushed to use mathematical algorithms known as value-added measurement to provide what proponents say is more precise quantitative information about a teacher’s performance. The local union, United Teachers Los Angeles, has fought back furiously and argued against any cookie-cutter approaches.

The two sides have made an uneasy compromise that has not eliminated potential problems that can arise from attributing a student’s performance on a test to a teacher’s abilities.

A deal ratified earlier this year allows the district to use state test scores and the Academic Growth Over Time measurement, which includes past test scores and demographics like family income, language and ethnicity. The deal specifically prohibits the use of individual teacher growth scores other than to “give perspective and to assist in reviewing the past CST (California Standards Test) results of the teacher.”

Critically, however, the agreement is silent on how much weight student data has. An announcement by Deasy in February that student data could account for up to 30 percent of a teacher’s evaluation was met with mixed responses.

Warren Fletcher, president of the 36,000-member United Teachers Los Angeles union, argued that the percentage violates the deal made with the union. It launched an offensive, equipping members with a “Teacher Evaluation Rights Toolkit.” Included was an objection letter to give to principals and examples of what the union said were permissible additions to this year’s initial evaluation planning sheets.

“We negotiated an agreement that honors the concept that administrators and teachers are adults and need to have an amount of discretion in how this works,” said Fletcher, who is in his second year as president of the union.

Despite her experience with the Academic Growth Over Time results, teacher Wynne Davis at Melrose Elementary said she welcomes the capped percentage, which allows for various types of student data.

“I don’t understand why anybody would be upset with 30 percent of your evaluation being tied to student achievement; that means 70 percent is not,” she said. “We are teachers, for God’s sake. I mean, how are you supposed to be judged if you are not using the kids’ work?”

New teacher evaluations

Although the use of test scores remains the most controversial piece of the evaluations, it remains to be seen how the main portion of the evaluations, which are based on intensive, time-consuming qualitative measures of a teacher’s performance, will affect schools and classrooms.

Los Angeles Unified began testing changes to the teacher evaluation system last school year with 425 teachers at 100 campuses who elected to participate and provide feedback.

This year, LAUSD’s pilot program was expanded to more than 1,100 school-site administrators who are working to fine-tune changes with about 900 teachers, one at each campus in the district. In the pilot, state test score gains in a particular class were analyzed and scored relative to the student’s predicted growth based on past scores, and relative to the average gains seen across the district and within different groups of the student population.

But the main focus of the pilot was to test the new qualitative measures.

Lengthy and more detailed evaluation score sheets are given to teachers for self-assessment and principals for observations. What was once a four-page evaluation is now 26 pages with descriptions of what an “ineffective,” “developing,” “effective” and “highly effective” teacher looks like in 61 areas; based on participant feedback, the district limited reviews to 21 areas this year.

Principals also are required to write exactly what they see and hear in the classroom as evidence and rationale of the teacher’s evaluation, a requirement that can leave observers tied to their computers and unable to take in the entire class experience, several pilot participants said. Because principals are working with one teacher as a part of the pilot, it’s unclear what the burden will be when they have to observe all of their teachers when the new system rolls out.

The pilot also tried to address the tricky issue of how to provide student achievement measures for the vast majority of teachers of grades and subjects that are not tested on a state exam, including physical education and art. For now, those teachers, particularly in the early grades, must cobble together a variety of measurements to give some sense of their performance.

At Melrose, veteran teacher Sachiko Miyaji is in her 17th year teaching. She and Wynne Davis were among five teachers at the math, science and technology magnet campus last year to volunteer for the pilot program.

The addition of the hotly contested student test scores in the pilot didn’t affect Miyaji because students take their first California Standards Test in the second grade. As such, the district can’t issue her and other second-grade teachers the individual Academic Growth Over Time scores it has been issuing since spring 2011.

Without state test score gains available, Miyaji relied on other student data for her evaluation. She included her students’ scores on district literacy tests and the University of Oregon’s DIBELS tests, short for Dynamic Indicators of Basic Early Literacy Skills, given three times a year at 20 percent of elementary schools in the nation.

But she said none of that measures the vibrancy of what’s actually happening in the classroom.

On a sunny Friday morning in March, Miyaji gathered her second-graders on the carpet at the front of the class for a lesson on lions and tigers.

“Did you know the back feet of lions and tigers have only four toes?” Miyaji asked. “So which feet? The front or the back?”

Her students, a rambunctious group of 7- and 8-year-olds, respond in unison, “The back!”

Reading from a book, Miyaji told her students that lion prides typically include two dozen cubs.

“That’s 24! Twenty-four cubs,” one student shouted.

“How do you know it’s 24, Carlos?” Miyaji asked.

“Because a dozen is 12,” Carlos said.

It’s observations like this that many teachers hope will provide a clearer picture of their classrooms and make up for any deficiencies in using test scores to judge their performance.

Examining student test scores

Nationally, experts have disputed whether student test score data can be used reliably to measure teacher performance – but nearly all agree that such data should be coupled with more qualitative information to ensure fairness and make sure the evaluations are useful to teachers.

Proponents say the test data is appropriate because it’s one way to measure what’s supposed to be the ultimate outcome of teaching – whether students are learning. The data also can provide a counterweight to principals’ observations, which, depending on how they’re structured, also can have reliability problems.

States and districts mostly have opted to look at student growth, as opposed to raw test scores, because raw scores can disadvantage teachers with large numbers of low-income, limited-English or special needs students, who tend to score lower on standardized tests.

Critics of growth measures have said they can fluctuate depending on the variables and number of years of testing data added to the mathematical formula, making their validity suspect, and worry that grading teachers based on tests will cause them to focus more on test prep in their classrooms.

Drew Furedi, formerly the executive director of Los Angeles Unified’s talent management division and now its executive director of human capital initiatives, said the piloted changes focus on evaluation as “a growth and development tool rather than a punitive tool,” which will tie results to “aligned, targeted, professional development opportunities for teachers.”

In a letter to teachers May 24, Deasy said the district will fully implement the evaluation changes tested in the pilot program next school year.

There’s another reason for the district to be motivated. Los Angeles Unified hopes to opt out of federal student accountability sanctions coming its way in 2014 under the No Child Left Behind Act. The Obama administration offered waivers from the law’s requirement that states steadily increase the number of students graded proficient on standardized exams to 37 states that agreed to other accountability measures, including new evaluations for teachers and principals.

With the promise of local accountability and coordinated efforts to improve, the district is one of nine in a consortium called the California Office to Reform Education – which includes Oakland, Sacramento and San Francisco – seeking a waiver from the U.S. Department of Education. If approved, it would be the first district-level waiver obtained in the offer intended for states.

Other districts and states also have responded to the Obama administration’s urging – through No Child Left Behind waivers and grant competitions like Race to the Top – that districts and states overhaul their evaluations. According to a review by the National Council on Teacher Quality, a Washington, D.C.-based reform group, 20 states in 2012 required student achievement as a significant part of judging teacher performance, including multiple states where student data accounts for 50 percent of an evaluation. In 2009, four states required student data in a significant way.

In many of those places, policymakers have encountered similar resistance from unions and educators on the front lines.

Teacher Denise Casco at Vista del Valle Dual Language Academy in San Fernando said the inclusion of student achievement could disproportionately affect her because she serves a more disadvantaged community than others in the district. She said she worries her students are more likely to lag behind on tests.

Nearly all 500 students on campus in the far northern part of the district are socioeconomically disadvantaged, and half are English-language learners.

“The playing field is not even,” said Casco, who is in her ninth year of teaching. “The funding is not even; the level of support is not even. So I am very cautious at putting a number or percentage for a particular level of student achievement in a teacher evaluation process.”

But not all of the changes are concerning, she said. Like many teachers around the country, she has embraced the more intensive classroom observations, which tend to offer teachers more specific feedback than previous versions.

As a kindergarten teacher last year who participated in the pilot program, Casco had a videotaped math lesson that was selected by the district as a model to train administrators on what to look for during an observation.

In it, her 5- and 6-year-old students are working in small groups. Students play a series of games related to frogs and fish to practice adding and subtracting and eventually are asked to illustrate a story with an equation and grade their own performance on a rubric.

“It forces you to reflect,” Casco said of the evaluation changes, which include the principal asking teachers after the observation if they achieved their objective. “It did hold me more accountable to the results than the old evaluation system would have.”

Lawsuit seeks to strike some teacher protections

The fight in Los Angeles has had a ripple effect across the state. Student plaintiffs filed a lawsuit last May against the state of California and several local districts, including Los Angeles Unified. The lawsuit aims to increase the impact of teacher evaluations by striking down multiple protections teachers have under the law.

Represented by Los Angeles law firm Gibson, Dunn & Crutcher LLP, the plaintiffs allege teacher protections such as tenure, seniority rules in layoffs and other teacher dismissal statutes disparately keep ineffective teachers in the classroom in violation of the state constitution’s equal protection clause. The case also cites court opinions that have held that “the right to an education today means more than access to a classroom” and that the state “has broad responsibility to ensure basic educational equality.”

Deasy and Xavier De La Torre, Santa Clara County’s superintendent of schools, have endorsed the case, according to the nonprofit Students Matter, the advocacy group sponsoring the lawsuit. After failed efforts by the state to get the case dismissed in the last year, a judge granted a request by the California Teachers Association and the California Federation of Teachers to join the lawsuit as defendants May 2. A trial date is set for Jan. 27, 2014.

Los Angeles Unified parent Lauren Campbell has a child plaintiff in the case and said that even though efforts are underway to overhaul the evaluation system in the district, it won’t do much to protect effective, less-senior teachers who she says she has seen get layoff notices year after year at Lanai Road Elementary School in Encino.

“I think that if we value education and we say we want great teachers for our kids, we need to make that job attractive, and part of the way to do that is not fire the best ones every year,” Campbell said.

Los Angeles Unified parent Lisa Liss also argues that the layoff system targets great teachers over less effective ones.

“The impact of having a really bad teacher goes way beyond one academic year. Those kids end up behind for years to come and maybe even permanently,” she said, “so every child should have a great teacher.”

Liss said you have to go to the top to get the wide-reaching changes needed.

“Individual changes will take forever,” she said. “To make a wholesale change at a much higher level is going to be much more effective and much quicker, and these kids don’t have much time to waste. Their education is happening now, not in 20 years.”

Ashly McGlone produced this story for The Hechinger Report and The Center for Investigative Reporting. Sarah Garland of The Hechinger Report contributed to this report.

“Did you work on any art projects?” Dieker asked Maria, a girl bent over a textbook in the back row.

“Yeah,” Maria answered.

“And what were you working on?” Dieker said.

“Just some sketches that I’ve been doing.”

The TeachLivE classroom simulator lets education students get the feel of managing a classroom. The virtual students respond to the teacher’s questions and movements and each student has a distinct personality. (Photo by John O’Connor/StateImpact Florida)

It sounded like a typical way to kick off a Monday morning class. There was just one catch: Dieker’s five students weren’t real; they were avatars projected on a screen. She talked to them through a headset and they responded. As Dieker, a professor at the University of Central Florida’s (UCF) College of Education, moved around, her view of the virtual classroom changed.

Started ten years ago, the so-called TeachLivE lab was developed by faculty in the education school at UCF, and at least 22 other universities across the country have opened their own labs using TeachLivE technology. Much like a flight simulator trains pilots, faculty use the virtual classroom to train teachers-to-be by helping them isolate and master strategies like higher-level questioning or behavior management.

“We can take those parts of teaching just like you could in a flight simulator, and you could work on just the landing or the takeoff or just maintaining your altitude,” Dieker said. “We can break those tasks down.”

Dieker and her colleagues estimate that―just like in a flight simulator―10 minutes in the simulator is equivalent to one hour in the classroom. Teachers-in-training can practice a specific lesson or skill over and over again. Studies conducted by UCF faculty have shown that after four 10-minute sessions in the simulator, teacher candidates have changed the way they operate in the classroom.

“We didn’t put any kids at risk while our teachers learn,” Dieker said. “We’re really hoping to make a first-year teacher look like a second-year teacher before they get started.”

(Disclaimer: TeachLivE receives money from the Bill & Melinda Gates Foundation, which is among the many funders of The Hechinger Report.)

Right now, UCF faculty members who want to bring classes into the lab must submit lesson plans two weeks in advance so Dieker and her colleagues can program the system to make sure the avatars make mistakes.

“When we get a request for a lesson on multiplying fractions … then we need to make sure that our students make the errors that are typical,” said Michael Hynes, director of the School of Teaching, Learning and Leadership at UCF’s College of Education. “So [the teacher candidates] know they can react to them.”

The software collects data during each training session, tabulating how much time the teacher spent talking to each student. It also records how the teachers responded to certain behaviors so that teachers can review their reactions afterwards.

If teacher candidates are not using good classroom management techniques, students might start to snicker or take out cell phones. Even though the class is small, it’s possible to lose control of students quickly

The avatars, always the same students Dieker was talking to, are fed information on a daily basis― weather, pop culture information, results of sports games―to help them carry on realistic conversations. Each of the avatar students has a distinct personality meant to mirror what teacher candidates might find in the classroom. Sean is the classic overachiever; he’s got an extra four pencils lined up on his desk “in case I break one.” CJ is more difficult, slouched in her chair and more reluctant to participate.

Right now there are only the five middle-school avatars, but those in charge of the virtual classroom said eventually they would like to expand to different grade levels as well as into principal training.

“Five years from now, I hope we’ll have 200 kids and you’ll call in and say ‘I would like a bilingual classroom with French and Spanish,’ ” Dieker said. “We would plop in third-grade kids [or] eighth-grade kids and ninth-grade kids, and people can customize the system.”

“It would be me, a computer screen and a phone,” he said. “I didn’t get that human interaction that I craved.”

University of Central Florida elementary education students discuss how to incorporate books, maps, magazines and other materials into lesson plans. (Photo by John O’Connor/StateImpact Florida)

So Clevenger changed track. He was accepted in Nova Southeastern University’s undergraduate teacher training program. On a campus tour, talking with professors and seeing the level of interest they seemed to have in the teacher candidates, Clevenger was sold.  He graduated from Nova in November with a degree in secondary social science and is now teaching world history at a high school near Nova’s Fort Lauderdale campus.

Although he switched from a tough major to one that has a reputation of being easy, he stressed that – despite what some people assume―the decision was not because he wanted to earn easy As.

“A lot of students see going into the education world as a fallback…That’s where you get the bad teachers,” he said. “It’s definitely not easy. It’s not something you wake up and do if you’re not passionate about it.”

A national push to improve the quality of teachers has focused largely on those already in the classroom, with the adoption new teacher evaluation systems and efforts to help struggling teachers and push out those who don’t improve. But increasingly, reformers who believe better teachers   will lead to greater student achievement are eyeing how teachers are trained in the first place—and finding training programs lacking.

For many of these critics, the problem with teacher education starts even before the first class begins.  These critics argue that low-quality students are recruited to education schools, drawn by low admissions standards and perceptions of education schools as a fallback option.  And high-quality candidates are being driven away from the field by school budget cuts imposed during the recession and the vitriol that often surrounds the education reform debate, educators say. Aware of their reputations, education schools find themselves doing a balancing act between boosting admissions standards and being able to fill seats.

“If they want to make [admissions] harder, I’m all for it,” said Les Potter, chair of the School of Education at Daytona State College. The college has open enrollment, but students must earn a 2.5 GPA to be admitted into the teacher training program. “But where are we going to get the students?”

The National Council on Teacher Quality (NCTQ), a research and advocacy group, has led the charge against low admission standards, frequently citing a 2007 McKinsey report that claims the majority of U.S. teachers are recruited from the bottom two-thirds of their class. By contrast, in countries such as Finland and Singapore that perform well on international measures of academic achievement teacher candidates are drawn from the top quartile.

NCTQ has been a constant critic of teacher training programs, producing annual reports to highlight what the organization deems as shortcomings in teacher training. In Florida, NCTQ found that only 23 percent of programs either require a 3.0 GPA before entry or are housed at a selective institution, roughly the same as the national average. (The study excluded the smallest programs.) Just five of Florida’s 42 education schools required a minimum ACT or SAT score for admission into an undergraduate program in 2012.

Nova, after nearly a decade of building up its undergraduate education program, saw its enrollment slipped from 692 in 2011-2012 to about 500 in the 2012-2013 school year. Despite that, Nova’s Fischler School of Education, into which students matriculate as juniors, is raising its entry requirements from a 2.4 in freshman and sophomore classes to a 3.0 for those matriculating in the fall of 2014. Incoming students will also need a 1,000 on their SAT.

“We’re very conscious that the people we’re awarding with our degrees are responsible for others’ children,” said Terry Davis, NSU’s director of undergraduate enrollment and recruitment. “The bottom line is important … [but] it’s not just, ‘let’s get everybody in.’”

Arthur Levine, president of the Woodrow Wilson Fellowship program, which provides an alternative pathway to teaching for career switchers, said Nova is taking a positive first step. He noted, however, that his research has found 41 percent of all college students have a 3.0 GPA or higher. “You have to ask, what is it based on? Is that enough?” he said.

In 2006, Levine, the former president of Teachers College, Columbia University, published a 140-page report on the state of teacher education with chapter headings like “The Pursuit of Irrelevance.” The study lambasted the state of education schools, including their admission standards.

Traditionally, education schools have been considered the “cash cows” of universities – low-cost, easy-to-fill programs that earn the institutions money. Levine’s work found that more than 40 percent of principals and education school faculty members thought the schools had low admission standards. Other research has shown that low-skill students in business and math majors are likely to switch to education, while high-skill education majors tend to leave those programs.

But there is conflicting information about who actually enrolls in education schools. While SAT takers who plan to major in education score well below would-be majors in many other disciplines, including engineering and psychology, many of these students never matriculate into education programs. Of those who pass the Praxis I, a basic skills test required for entry into teacher training programs in 30 states, the SAT scores of would-be elementary school teachers are below the national average, but those of secondary school teachers are at the average, according to Levine’s report.

Founded in 1964 as a graduate school, Nova was soon focusing on distance education and, later, it expanded those offerings to online courses. Now, universities across the country are trying to increase the number of online courses they offer, and online teacher training programs are part of the explosion. Walden University and the University of Phoenix, both for-profit institutions that offer extensive online coursework, are now the top producers of teaching degrees in the country, according to federal data.

Although still maintaining its online and graduate programs, Nova’s primary push is to build a robust set of brick-and-mortar undergraduate programs and provide a traditional campus experience for its students. The ultimate goal is to become a selective research university, competing for the best students in Florida and across the country.

Nova’s main undergraduate campus in Fort Lauderdale is full of brand new buildings. The $92 million campus center, opened in 2006, features a food court, student lounge and a rock climbing wall. Next door, construction is underway for a new collaborative research center.

Even so, after years of growth, the number of in-person undergraduates is faltering, including those who want to be teachers.

“There’s a lot of issues, a lot of challenges,” said Jamie Manburg, Nova’s executive director of teacher education and undergraduate programs. “I’m just not seeing, in my opinion, the demand that we used to see.”

Manburg attributed the drop in enrollment to worsened job prospects for teachers compared to other majors. The upside, he said, is that a lack of job guarantees means that those who chose to enroll in teacher training programs now are more likely to do so because they want to make a difference in children’s lives.

The shrinking number of teacher candidates comes at a difficult time. With subjects like math, science and special education facing perpetual teacher shortages and a wave of baby boom teachers poised to retire, student enrollment in Florida continues to increase, to more than 2.69 million students this school year, from less than 2.63 million in 2008-09.

There has been an increase in the number of Florida programs training teachers but that means  the competition to recruit education students across the state has become stiffer. In the 2000s, the state supported districts developing alternative certification programs to target shortage areas, like special education, math and science. In 2008, the legislature passed a law allowing community colleges to offer bachelor’s degrees – including in education – to address degree areas where four-year schools could not meet the demand.

Daytona State Community College already had an education department that offered several pre-education major courses. It jumped at the chance to hold on to its students for the full four years, rather than losing them to another institution. In 2009, the school dropped community from its name and opened several bachelor’s programs, including one in education. It graduated its first class of teachers in December 2010.

Each spring, Potter, Daytona’s education program chair, visits the school’s freshman and sophomore pre-education classes, making a pitch for staying in his program rather than transferring to a larger school like the University of Central Florida. “We take a real personal touch with everyone,” he said. In particular, Potter tries to persuade undecided students and potential math and science majors to consider majoring in math or science education, or in another shortage area, like special education.

As Daytona works to grow its program, Nova is focused on retooling its offerings. While admission standards are going up, Nova administrators are streamlining graduation requirements, eliminating some and embedding others in the classroom. For instance, Nova is considering removing an assignment requiring each student to develop a portfolio of their college work, including lesson plans.  Administrators are also discussing how to make the school more flexible.

“We have a generation of students who want things very quickly [and] want to customize their programs to a certain extent,” Manburg said. “We’ve got to respond to their needs and we’ve got to change our programs.”

In part, Spalding was providing content to her students by introducing them to materials they might use – like National Geographic magazines and the poetry books. But she was also modeling teaching strategies, like small group learning, and introducing activities, like the timeline exercise, that she hoped her students would someday mimic.

University of Central Florida education professor Lee-Anne Spalding uses an interactive white board to shows students how to connect a drill using coins to both math and history. Critics say education programs, such as the one at UCF, have few standards for entry and do not adequately prepare graduates to lead a classroom. (Photo by John O’Connor/StateImpact Florida)

“You are more likely to use the instructional strategies I’m proposing to you if you actually do it,” she told her students.

UCF is the largest producers of teachers in the state; the university’s education school enrolls more than 2,000 students. It prides itself on being one of the strongest—if not the strongest—teacher training program in Florida, a position it has gained, school officials say, by nimbly responding to changes in the profession. But there is no real way to test that claim. The university, like many education schools across the country, often must rely on anecdotal evidence from principals and graduates to determine that its programs are working, rather than hard data showing students are performing better.

Conventional wisdom holds that many, if not most, education schools are doing a poor job at training teachers; after all, they have a history of taking in some of the lowest performing students, and student achievement in the United States has stagnated. Nationally, education schools have been criticized for being far too easy and, as a result, pumping ill-equipped teachers into the system and harming student achievement. Schools across the country are trying to mitigate the criticism by changing curriculum or increasing the amount of field experience teachers receive.

Florida and several other states are also creating accountability systems so education schools will develop quantitative ways to measure their programs’ success. But for now, teacher preparation remains over-saturated with options―undergraduate degrees, master’s programs, in-school residencies and online courses―that provide little evidence of their effectiveness. And as thousands of Florida’s baby boomer teachers prepare to retire, there is little consensus about how to best train the next generation of teachers.

“I don’t know of any other profession that has this kind of uncertainty about the kind of preparation needed,” said Arthur Levine, president of the Woodrow Wilson Fellowship program, which provides an alternative pathway to teaching for career switchers.

The bulk of teachers are still trained in traditional undergraduate colleges of education, which have borne the majority of criticisms. In particular, Levine and others have argued the schools are not rigorous enough and don’t focus enough on the subject matter content—like geometry concepts or Shakespeare—that teachers need to know in order to pass on the knowledge to their students.

The very idea of an “education degree” may be an antiquated concept, says Timothy Knowles, director of the University of Chicago Urban Education Institute. He argues that there is little evidence to show that traditional programs’ focus on pedagogy—including classes on child development and how students learn—helps new teachers succeed in the classroom.

“Maybe we should ask some deeper more existential questions about the value of teacher education as it is constructed,” he said.

One study, by Cory Koedel of the University of Missouri, found that undergraduate education schools tend to give higher grades to students than other departments, a finding supported by data that The Hechinger Report collected from the University of Central Florida.

Of UCF’s 65 departments, just six, including three small programs run through the dean’s office and the Florida Interactive Entertainment Academy, a graduate video game design program, gave out a higher percentage of As in their classes than the School of Teaching, Learning, and Leadership. According to the UCF data, 73 percent of grades awarded in these teacher training courses from fall 2011 to summer 2012 were As or A minuses, compared to 34 percent in electrical engineering courses and 40 percent in food services and lodging management.

“Students are graded individually based on their mastery of professional knowledge and skills; there’s no grading on a curve,” UCF spokesperson Courtney Gilmartin said in an email. “If faculty members do their job well … every future teacher demonstrates their competencies to the highest level and graduates with the knowledge and skills required to become a highly-effective classroom teacher.”

Many education schools across the country similarly argue that grades are a positive reflection of those enrolled in an education school, not a condemnation of them. A good grade doesn’t mean it wasn’t earned, said Mike Rosen, an education student at Daytona State College. “The assignments are not easy,” he said, noting that some keep him up until the early hours of the morning. “But every single one of them is necessary.”

As an example, Rosen pointed to an assignment in a children’s literature class: he’d been asked to set up his own mock library for an elementary school classroom. He visited schools and interviewed teachers about the libraries they’d designed before choosing the books he would include in his and how he would arrange them.

Arthur McKee, managing director of teacher preparation studies at the National Council on Teacher Quality (NCTQ), a nonprofit advocacy group that has been one of the most vocal critics of teacher preparation programs, says he doesn’t believe that the high GPAs of education students can be explained by excellent professors or extremely dedicated students. “We think it’s much more possible that the teacher preparation programs are just not holding the candidates themselves to a high enough standard,” he said.

NCTQ has pushed for aggressive education reforms across the country and have targeted teacher preparation programs as institutions in need of drastic changes. This summer, group, in partnership with the U.S. News and World Report, plans to release a highly controversial set of ratings for teacher training programs based on the syllabuses of classes offered at schools of education. Critics of the group’s methodology say the focus on coursework won’t solve the problem of figuring out which schools are producing the best teachers and which aren’t.

Sandy Robinson, dean of UCF’s education program, noted that grades are not the only factor in determining if a student graduates. Before exiting the teacher training program at UCF, students must spent 800 hours in a classroom. Regardless of their grades, they may be counseled out of the education program if they don’t perform well during their internships or student teaching.  “That’s an important part of the responsibility we have,” Robinson said.

Many education school critics say in-classroom experience should be an essential if not the main focus of a teacher training program. Florida requires 10 weeks of full-time student teaching in order to complete a traditional education program. Teacher candidates must also “demonstrate they can make a positive impact on student learning,” Kathy Hebda, a deputy chancellor at the Florida Department of Education, said. But the law does not specify how that impact must be measured.

Many groups are becoming increasingly concerned about whether it’s possible for programs to demonstrate such impacts from their graduates. Policymakers, including officials in Florida, have turned to student test scores as a means of evidence. But for now, test scores only capture a small fraction of the teachers trained in education schools–although Florida is moving toward end-of-course assessments for all grade levels and subject areas. There are also concerns about the reliability of using standardized tests as a gauge of teacher effectiveness.

Even so, the Council for the Accreditation of Educator Preparation has released a set of contentious new standards, under which programs would have to prove that their graduates were able to raise student test scores. The U.S. Department of Education is working on new requirements for teacher preparation programs to increase accountability based in part on “student learning outcomes.”

At the state level, Louisiana has led the way in searching for ways to measure the performance of preparation programs. In 2007, the state passed a law requiring teacher education programs be assessed, based partially on how the students of their graduates performed on standardized tests. The results have prompted some schools to significantly change their programs, but no schools have been shut down or punished by the state so far. A dozen other states, including Florida, also use or plan to use student test scores to rate teacher training programs.

In 2009, Florida began releasing rankings of education schools based on what percentage of a teacher’s students passed the state standardized tests. The state, which promised to improve education school accountability in its Race to the Top grant, has since stopped publishing the results in anticipation of the state’s new teacher evaluation process, which will use student test scores to rate teachers. Instead Florida is developing a new rating system for education schools that covers six areas including student achievement, graduate employment and retention.

The new system won’t differ much from how the state currently oversees teacher preparation. As is now the case, programs that are denied state approval under the new ratings will be given time to improve. But even if they fail to do so, programs will not be shut down. The only consequence will be that the transcripts of the program’s graduates cannot say they completed a state-approved program.

Many education schools say that, regardless of the new requirements, they are always actively seeking to improve. UCF, for instance, has added more classes in reading instruction and English as a Second Language. The school is looking for ways to independently measure the impact of its graduates from all of its programs.

At UCF and Daytona State, faculty members are required to spend a day in a school at least once a semester to keep their pulse on what’s current in education. Both schools, and many others, do principal and alumni surveys, tweaking their courses based on the results.

“Like any entity, we want to get better,” Robinson of UCF said. “We understand that our existence depends on the viability of the graduates we produce.”

Yurica and Jones are teachers-in-training with the Urban Teacher Residency, a partnership between the Los Angeles Unified School District and four southern California universities, which provides an alternative route to the classroom.

Landon Yurica and Alycia Jones, part of the Urban Teacher Residency program, watch their mentor teacher deliver a geometry lesson at Washington Preparatory High School. (Photo by Jackie Mader)

The program takes three semesters compared to an average of six semesters in traditional programs for students who start as undergraduates, and two for post-baccalaureate programs. It also demands a commitment of at least three post-preparation teaching years from its participants. It is one of an expanding pool of alternative programs capitalizing on the belief that the more experience an aspiring teacher has in a classroom, the better. The number of alternative programs nationwide has skyrocketed, rising from 70 programs in the 2000-2001 school year to 658 in 2011, according to the U.S. Department of Education, and these programs now make up 31 percent of all teacher preparation programs in the nation.

Yet experts on teacher preparation acknowledge that little is known about which strategies actually work best for developing high-quality teachers. In 2008, James Wyckoff, a professor of education and public policy at the University of Virginia, was one of several researchers who looked at components of teacher preparation programs in New York City to determine which seemed to impact student achievement the most.

“I think what is remarkable is how little we know about teacher preparation,” Wyckoff said.

His study found, however, that one feature that can make a difference in outcomes for students is the amount of time aspiring teachers spent engaged in meaningful work in classrooms before they graduate from a training program.

In California, as in many states, the number of hours required for student teaching varies greatly by program and the state has no minimum. Some schools, such as Loyola Marymount University, require as many as 1,600 hours, according to the California Commission on Teacher Credentialing. Others, like Chapman University and Fresno Pacific University, require less than 500 hours. A handful of programs require no more than 200 hours. Nationwide, traditional teacher preparation programs required an average of 514 student teaching hours during the 2008-09 academic year, according to the U.S. Department of Education, far less than the average of 901 required that year by alternative programs that are not based out of universities.

Emily Feistritzer, president and CEO of the National Center for Alternative Certification, says that nationwide, alternative programs tend to place aspiring teachers in the classroom from the very beginning, so these numbers naturally would be higher. Those new teachers are often the teacher of record immediately. By contrast, in traditional training programs, students observe and then are observed by a mentor teacher. “Student teaching, that terminology, has very little relevancy in the alternative routes,” Feistritzer said. “An alternative route program is generally a field-based program.”

What constitutes an alternative route varies widely, however. Every state determines its own definition for alternative programs, meaning a program that one state has classified as alternative may be classified as traditional in another state, despite having many of the same characteristics. One example is Teach For America: In some states it is considered a preparation program, in others, a recruiting organization.

In California, alternative programs are called “intern programs” by the state, and refer to programs where participants teach in classrooms during the program, usually as the teacher of record. And most so-called alternative routes are actually run by traditional university programs, although that may be changing.

The Urban Teacher Residency Program falls into a small category of alternative programs in California run by school districts. These programs tend to have partnerships with local universities to offer education classes to participants, but emphasize time in the classroom as a crucial component of the training. Many of these programs were created to address teacher shortages in specific subject areas, or to attract candidates who historically have been underrepresented in the teaching force, such as males or minorities. Others were created in the hopes of developing better teachers, either through the program’s methods of training teachers, or by attracting candidates with subject matter expertise, like those with degrees in math or science.

Between the 2008-09 and the 2009-10 school years, the number of students in both traditional and alternative teacher preparation programs in California dropped, most likely due to lack of job security, educators say. But according to the U.S. Department of Education, alternative programs based at universities across the country saw a 3 percent increase in the number of people completing them during that time. Alternative programs not run by universities, such as school district programs, saw an 18 percent increase.

The increase is likely due to more programs and candidates embracing the idea that time spent working in a classroom is more beneficial than time spent reading a textbook about teaching. “We find that the closer you get to the classroom, the teacher training is better,” Feistritzer said.

Alternative routes may also be more convenient—and less expensive—than a university. The Urban Teacher Residency program at California State University Dominguez Hills pays its participants a stipend of up to $20,000 over the course of the 18-month residency, while others offer perks like free master’s degrees. (At California State University, the graduate and credential programs cost about $6,800 per year for state residents; private schools like Loyola Marymount University can cost upwards of $30,000.)

But just as little is known about the effectiveness of traditional routes, there is little evidence that alternative routes are doing a better job of effectively preparing teachers.

During the 2009-10 school year, teachers prepared through an alternate route accounted for 10 percent of those attempting to pass a performance assessment in California, a requirement before earning a credential. These teachers also had the lowest pass rate on their first attempt to take the exam of all candidates.

California’s Commission on Teacher Credentialing says that teachers in some routes take parts of the exam early in their program, however, perhaps accounting for lower pass rates than those who take the exam at the end of the preparation program. Different versions of the test also have lower pass rates than others.

The Urban Teacher Residency program, begun in 2009 to fill critical teacher shortages in urban Los Angeles schools, has embraced the limited research suggesting that more classroom experience, especially when it replicates what teachers will be expected to do in their own classrooms post-graduation, produces better teachers. Teachers in the residency program spend an average of about 1,300 hours in the classroom in student teaching, more than the average number completed in nearly 90 percent of the alternative programs in the state.

Unlike some alternative programs, though, the program gradually introduces candidates to full-time teaching. Teachers-in-training, called “residents,” spend a summer semester taking classes through a university partner, then immediately enter a classroom to become acclimated. For one semester, the residents observe an experienced teacher nearly full time. The residents say this allows them to build relationships with students before the second semester, when they begin teaching a few classes on their own with mentoring from a more experienced teacher.

Jones, who is in her third semester of the residency program, says that she doubts she could learn the same lessons about managing a classroom and keeping students engaged through courses at a university. “You can’t talk about it, you can’t have conversations about it, you can’t see videos about it,” she said. “You actually have to see it and you have to be in it.”

Alternative programs are often designed to address another frequent weakness of traditional programs. Both nationwide and in California, schools of education graduate an overabundance of elementary school teachers. The Urban Teacher Residency program is focused on producing only teachers who will fill some of the shortage areas that have plagued California schools for years: math, science, and special education. Residents are placed in urban, low-income schools and teach only secondary math or science.

To ensure the program is meeting school district needs, HR administrators of the Los Angeles Unified School District sit on a selection committee to interview candidates for the program, and the two entities share data frequently.

“They need information from us, and we need information from them,” said Kamal Hamdan, program director of the Urban Teacher Residency at California State University Dominguez Hills.

Specifically, the district shares student achievement data with the residency program, which Hamdan says is crucial for determining how effective the program graduates are in the classroom, and ultimately, how the program can help. “It shouldn’t be only the district’s obligation,” he said. “It should be our obligation to step in and say, ‘wait a minute, this teacher might be struggling, what are we going to do as an institute of higher education?’”

There’s not enough data yet, however, to show that the residency program is producing high-performing teachers who outshine graduates from traditional routes, something Drew Furedi, the executive director for talent management at the Los Angeles Unified School District, acknowledges.  “We’ll see if that’s a model that prepares people in a demonstrably different way,” he said.

By the end of the first page, Martinez had already learned an important lesson: “Beware of redundant problems,” he scribbled on the side of his paper before flipping it over to finish the problems on the back.

Mario Martinez, a graduate student in California State University Northridge’s teacher preparation program, examines a high school algebra test he created for a class assignment. (Photo by Jackie Mader)

Martinez has until the fall to hone his skills before he will be sent into a classroom to practice as a student teacher. And he has at least a year before he will have to prove that he can not only teach math, but also create tests and analyze student results. It is a skill that many educators say is a sign of a good teacher, and one so important it was included in a lengthy exit exam that all aspiring teachers must take before they receive a teaching credential from the state.

Aspiring teachers videotape themselves teaching a lesson and write several lengthy reflections. California introduced the performance assessments in 2001 to adhere to a 1998 state law. Teachers must pass them in order to receive certification.

Every teacher preparation program in the state must choose one of three versions for students to take, each of which centers around the teaching and self-reflection activity. The Performance Assessment for California Teachers, or PACT, is the test of choice for Northridge and more than 30 other teacher preparation programs in the state, and many classes, like Cheng’s math methods course, design curriculum around the assessment to ensure students are prepared to pass.

Although it is largely untested and debated amongst educators, the PACT has served as a model for a national exam, known as the edTPA, that at least 25 states are introducing. Developed by 12 California institutions in 2001, the PACT was put on hold when the state suspended the performance assessment requirement in 2003. Three years later, the requirement was reinstated, and in early 2007 the state’s Commission on Teacher Credentialing approved the assessment.

The multi-part test, which often takes a semester to complete and results in dozens of pages of essay reflections, tries to assess whether an aspiring teacher is able to teach multiple learners in real classrooms. It has been tapped as a nationwide model because supporters say it presents a complex picture of a candidate’s strengths, weaknesses, and classroom readiness.

But many educators hesitate to say that the new performance assessments are creating better teachers or that passing them is a sign a teacher will be effective, partly due to the lack of more evidence.

Martinez says that Cheng’s class has spent extra time on designing and grading tests for lessons they have created because it is typically “the part of the PACT that math teachers do the worst on.”  While some say this practice of designing teacher preparation curriculum around the PACT bears resemblance to K-12 teachers “teaching to the test,” many educators at Northridge say the PACT is focused on critical areas of good teaching, like planning lessons with strong student assessments, and modifying lessons for English language learners and students with disabilities, and that it therefore only reinforces what candidates should learn anyway.

“The PACT certainly has forced us to give greater attention to certain aspects of what it means to teach and to deliver a lesson more effectively,” said David Kretschmer, chair of the Department of Elementary Education at California State University, Northridge. “We are churning out a better product, if you use that expression, than before we adopted PACT.”

That was the intent of the creators of the PACT, educators from teacher preparation programs in California who wanted to ensure that all pathways to teaching in the state were centered on research-based teaching practices that would produce better teachers.

Some research has found that high scores on performance exams like the PACT may signify that a teacher will be more effective in the classroom. One study out of Stanford University, which helped design the PACT, found that for each additional point an English Language Arts teacher scored on the exam, which is scored on a 44-point scale, students averaged a gain of one percentile point per year on California standardized tests. But the study only looked at 14 teachers and their 259 students.

If passing the PACT means teachers are prepared for the classroom, then by pass rates alone it would indicate that programs using assessment are, for the most part, producing teachers ready for the challenges of the classroom. In the 2009-10 school year, 33 percent of aspiring teachers in the state applying for their credential took the PACT. Ninety-four percent of them passed all sections of the exam on the first try.

But according to the California Commission on Teacher Credentialing, the PACT’s pass rates are much higher than those on the California Teaching Performance Assessment (CalTPA), taken by a majority of teacher candidates in the state, and the Fresno Assessment of Student Teachers (FAST), taken only by candidates at California State University Fresno. The CalTPA had the lowest pass rate, with only 77 percent of candidates passing all sections of the exam on the first try. The FAST has a first-time pass rate of 87 percent.

The high pass rates have skeptics wondering if the performance assessments are rigorous enough. All three versions of the assessments are usually scored by the institutions themselves, and students can retake them if they fail the first time.

Linda Darling-Hammond, a professor of education at Stanford University who helped design the edTPA, the national test, says the high pass rate on the PACT is expected. Teachers in California take up to three standardized tests, including a basic skills assessment and several subject matter tests, even before they take the PACT or one of the other two versions of it.  Darling-Hammond says each exam knocks out about 10 percent of the aspiring teacher pool in the state. (In the 2009-10 academic year, 78 percent of candidates passed the state’s basic skills assessment, and 81 percent of applicants passed the reading instruction exam.)

“This [pass rate] is only the people who’ve made it through all those gauntlets, that managed to get into the program, and haven’t caved when they were asked to do the PACT,” said Darling-Hammond.

She added that the preparation programs that use the PACT, including the University of California system, Stanford, and several schools within the California State University system, have the highest selectivity in admissions to their preparation programs. “If this were statewide,” added Darling-Hammond, “the pass rate would certainly be much lower.”

A report from the state’s Commission on Teacher Credentialing cautioned against comparing the pass rates. Unlike the PACT, which is taken at the end of the preparation program, candidates take the four sections of the CalTPA at different times throughout their programs. Some programs counsel students out before they take the performance assessment, meaning only the top students may end up taking the exam.

Opponents of performance assessments say that preparation programs, and the state, are missing the point by relying on an assessment to determine if teachers are prepared for the classroom.

Ann Schulte, associate professor at California State University, Chico, says that preparation programs should be focused on working with and assessing teacher candidates in the field, so they receive frequent observations and feedback during their student teaching experiences from someone with extensive knowledge of their abilities and classrooms.

Schulte cited research that found alignment between the results of those who pass the PACT and the observations of educators supervising those candidates in the field. “It begs the question then, ‘why are we doing it?’” Schulte said.

Elsewhere in the country, some educators and students have asked the same question, and subsequently refused to administer or take the national version of the assessment. In 2012, all but one student in the secondary-teacher training program at the University of Massachusetts Amherst refused to participate in the exam, arguing that mentors who observed them in a student teaching setting for months would be better judges of their teaching ability than Pearson, the education company administering the exams.

More concerning to some schools is the idea that pass rates on performance exams could be used to determine the quality of teacher preparation programs. Since 1998, the federal government has attempted to increase the accountability for preparation programs by requiring states to collect and report information about the programs, including completion rates, average scores on state and national teaching tests, and the number of student teaching hours required.

California includes pass rates from performance assessments in its own annual analysis of this data, and uses that data as one of many measures that determines if a school of education is “low-performing.”

In California, there is general consensus that the performance assessment, which encourages students to focus on how they would teach a variety of students, has at least created more thoughtful teachers, even if the research isn’t clear that the tests are improving the quality of the teaching force.

“It’s hard to imagine that the exercise isn’t raising their expertise level,” said Julie Gainsburg, associate professor at Northridge. Gainsburg says that the assessment is requiring students to reflect on their teaching and planning in ways that are more sophisticated than before the PACT.

“Does PACT make a better teacher? No,” said Nancy Prosenjak, a professor at Northridge. “But I think we have a substantial program that’s research based, we have the PACT,” she added. “So with all of those, maybe we have better teachers.”

“You’re still smiling, this is good!” she told her students as the chatter died down. A few stragglers trickled in, wearily making their way to their seats.

The 17 students had spent the morning in classrooms spread across North Los Angeles and would devote the rest of the afternoon to discussing their experiences in Prosenjak’s supervised fieldwork course, a class dedicated to student teaching. The class is a requirement in the university’s post-baccalaureate teacher preparation program.

“Who taught for one hour?” Prosenjak asked.

Nancy Prosenjak, a professor at California State University Northridge, prepares to debrief with her students after their first day of student teaching. (Photo by Jackie Mader)

Nearly all students raised their hand.

“Who was in charge for more than an hour?”

Only five hands remained.

“How did that feel?” Prosenjak asked.

“It went quickly,” responded one student. “I liked it.”

For the rest of the semester, the students will gradually take over more responsibilities in local classrooms, many of which are in low-performing schools in high-poverty districts. Then, after a year of coursework, including an average of nearly 500 hours of practice in schools, most can seek out jobs running their own classrooms by this fall.

A high-quality teacher can make all the difference to a student who is struggling, according to a growing body of research that has found teachers are the largest in-school factor affecting student achievement. And there’s an emerging consensus that how teacher candidates are chosen and trained can make all the difference in developing teachers with the knowledge and skills to propel their students ahead.

But even after students leave schools of education, and after years of reforms, the institutions often have no way of ascertaining if their programs produced strong teachers. In 1998, when only 20 percent of the California’s fourth-graders tested at or above proficient in reading on the National Assessment of Educational Progress, lawmakers in California passed ambitious legislation meant to strengthen teacher preparation programs. The legislation allowed for multiple routes to the classroom and introduced uniform design standards for those programs. It also created new tests to ensure aspiring teachers were ready for the classroom.

Schools of education adopted the reforms and adapted their programs beginning in 2002. In California, there are various routes to becoming a teacher, all requiring attainment of a bachelor’s degree, passing several competency exams, and spending time in a classroom.  Yet nearly 10 years after the reforms, there is little more than anecdotal evidence—and no hard data—to show whether programs, and graduating teachers, are better than those who graduated before the reforms.  Student test scores, which are increasingly used to assess teacher performance, have shown little improvement. By 2011, the number of California students proficient on the national reading exam had increased only five percentage points, to 25 percent from 20 percent.

David Rattray, senior vice president of education and workforce development for the Los Angeles Area Chamber of Commerce, co-chaired a transition committee in the wake of the reforms, and says that there is still a need for changes throughout the arc of the process, from recruiting students to continuously developing experienced teachers. “Of course all of us are concerned with making sure what we put out there does become reality,” Rattray said. “But we’re also humble enough to know this is tough work.”

The need for quality teachers is especially urgent in California, where experts anticipate that thousands of teachers will retire in the next few years even as fewer people are attracted to the profession. (Between 2006 and 2011, enrollment in the state’s teacher training programs fell by 33 percent, most likely due to lack of job certainty, educators say.) The retirement figures, combined with a large number of teachers currently teaching in subjects  they are not certified in, and an ongoing shortage of teachers in areas like math, science,  and special education, have researchers estimating that California could  lack nearly 33,000 teachers by 2015.

The declining number of students studying to become teachers has forced programs to try new recruiting tactics, including expanding to online programs that can draw in students from rural areas or distant parts of the state. More new California teachers are also earning their degree through district-run programs where education students start teaching in classrooms almost right away, and take classes at a local university in the evening. But for aspiring teachers in California, enrolling in a traditional teacher preparation program through a private or public university is still the most popular route to the classroom.

At Northridge, Michael Spagna, dean of California State Northridge’s college of education, says that the school of education underwent extensive changes after the reforms were passed in 1998, which he says was a “seismic shift” for California.

Many say that the biggest change to teacher preparation was the introduction of a mandatory performance assessment, a multi-part exam meant to assess how prepared teachers are for the classroom. The exam is required for certification, and is taken either at the end of the program or at certain points during the program, depending on the version of the test the training program uses. Schools of education created classes solely focused on preparing students to pass the exam, which centers on the “teaching event” where teacher candidates videotape a lesson and analyze it in a series of lengthy essays.

In education classrooms across California, just the mention of the performance assessment elicits groans. “They think it’s this giant, big thing that they’re writing,” said Nancy Prosenjak of Northridge. “Actually it’s what teachers do every day,” she added. “But they just don’t write down 50 pages about it.”

Graduate students in the California State University Northridge teacher preparation program discuss their student teaching assignments. Before they graduate, students in the program will spend at least 500 hours in classrooms across the greater Los Angeles area. (Photo by Jackie Mader)

Programs were also asked to make uncomfortable changes. After the passage of legislation in 1970, students could no longer become teachers after only completing an undergraduate program. Schools of education had to shrink what had been multiple-year courses of undergraduate study into a year-long post-baccalaureate offering. And while aspiring teachers could still begin taking education courses in their undergraduate years, they now had to stay for a fifth year. When the 1998 reforms were passed, schools suddenly had to fit even more required coursework, such as health and technology education, into the year. The reforms brought an emphasis on teaching English language learners, which meant programs had to infuse strategies to reach these students throughout their courses.

“We were struggling,” said David Kretschmer, professor and chair of the Department of Elementary Education at California State University Northridge. “It was a matter of squeezing other things out.” The school discarded courses focusing on generic methods of teaching, instead offering methods courses specific to subject areas. Kretschmer says that many courses improved, and the emphasis on English learners has mostly been seen as a success. But other courses didn’t drill down as deeply as they used to. “That was just an untenable position, because we couldn’t do what we needed to do,” he said.

As schools of education tinkered with their courses and focused on preparing teachers for the new test, experts began to realize that there was no accountability system to make sure the reforms were working.

In 2006, Sharon E. Russell, a professor at California State, Dominguez Hills, published one of several reports that highlighted the difficulties in tracking the impact of the teacher preparation reforms and argued for creating a system to connect teacher performance with student achievement as a way to see if they were working.

Officials at teacher preparation programs say they are eager for guidance, and they point to flaws in the state’s current accountability system for teaching programs, which looks at factors like admissions requirements and class offerings before approving programs. Julie Gainsburg, associate professor at California State University Northridge, was part of a research team that in 2009 attempted to study the classroom performance of recent graduates. The team found that it was hard to disaggregate the teacher preparation program’s impact from other factors, like a teacher’s own philosophies about teaching, or professional development they receive while teaching at their school.

“Unfortunately we don’t know a lot about what happens to our graduates when they go out,” Gainsburg said.

Several other researchers from Northridge have attempted to study the performance of their teachers after graduation by using student test scores from the classrooms of recent graduates, however. In 2007, David Wright, the director of the California State University system’s Center for Teacher Quality analyzed how graduates from Northridge compared to those from other teacher preparation programs in the state by looking at student achievement data.

Wright reported that in reading, graduates from other programs tended to slightly outperform CSU Northridge graduates. But another study found that teachers trained by California State University programs appeared to be more effective at teaching math to English language learners than teachers trained elsewhere.

The Center for Teacher Quality has produced annual reports since 2010 that compare student test scores of teachers within various California State University campuses against those from other programs, but the center cautions that test scores must be supplemented with other data because California’s tests don’t completely measure all aspects of what a student has learned.

Debating the use of student data

Spagna argues that student test score data is the key to helping teacher programs—and the state—figure out whether they are succeeding. “No institution of higher education, no teacher preparation program, is ultimately going to be able to tell how successful they were without pupil learning [data],” Spagna said.

The problem is that while the college sends out surveys to graduates and employers, Spagna says it does not receive information from local school districts about how effective graduates are in their classrooms. “The right side of the equation is still missing,” he added.

Besides the surveys, programs can also look at the results of the performance assessments, which candidates take before receiving their credential. Teacher educators mostly praise the test because they say it helps them develop thoughtful teachers, but some question the rigor and credibility of the tests, which can be taken twice in California and which are scored by the institutions training the candidates. One of the performance assessments, taken by about 30 percent of all teacher candidates in the state, has a 94 percent pass rate for first-time takers.

And some say success on that exam does not guarantee a teacher will be strong. “It’s problematic,” said Gainsburg. “To imagine that this test given at [this] time…in their teaching career should correlate to what their kids are doing five years later, it’s so indirect,” she added.

California is not alone in grappling with how best to improve the development of new teachers. Elsewhere, education schools are under fire and also dealing with new competition, as online programs and alternative pathways vie for a shrinking population of people interested in becoming teachers.

In 2006, Arthur Levine, the former president of Teachers College, Columbia University, published a lengthy report on the state of teacher education, calling it a “troubled field” and criticizing schools of education for having low admission and graduation standards, and “wide disparities in institutional quality.”

Students in the undergraduate program at California State University Northridge take notes as their classmates practice lessons they planned about phonics. In California, students who begin teacher preparation at the undergraduate level typically must stay a fifth year to complete student teaching and other state requirements. (Photo by Jackie Mader)

A national debate has raged for the past few years about whether student test scores can provide a reliable and fair measure of teacher performance. Using those scores to examine a teacher’s academic training is also complicated.

In California, experts say it is difficult to tie a teacher’s performance directly back to the school they attended, in part, because another aspect of the 1998 reforms required teachers to receive additional training on the job. “There are a lot of factors that go into a teacher’s performance in the classroom, and certainly some of those do happen after teachers leave the preparation program,” said Sarah Almy, director of teacher quality at the Education Trust, a Washington D.C.-based advocacy group that pushes for more accountability in education.

James Wyckoff, director of the Center on Education Policy and Workforce Competitiveness at the University of Virginia, agrees that it can be complex. But he says that some researchers have found that it is helpful to compare the success of teachers from different programs. “The information we’re getting from this is better than nothing, which is sort of what we’ve had before,” he said.

Other states have increasingly embraced the use of student test scores for measuring teacher programs. Louisiana has used student test score data since 2006 to determine which teacher training programs are most effective.  While some say it has raised accountability for schools of education, some education schools have pointed to flaws and called the systems unfair.

The federal government has also tried to regulate quality in teacher training. In 1998, the same year California passed its reforms, Congress passed a new version of the federal Higher Education Act that required states to identify, report, and help low-performing teacher preparation programs.

But like California’s law, the impact of the requirements still isn’t clear more than a decade later. Each state can determine its own criteria for evaluating programs, and in the past decade, only 25 states have identified a program as “at-risk” or “low-performing.” And among the 42 states and the District of Columbia, which provided a detailed description of their criteria to the federal government, 17 states and the District of Columbia used only a single criterion to evaluate teacher preparation programs, such as the program’s completion rate or its pass rate on state certification assessments.

Recruiting the best and brightest

At 12:15 on a recent Tuesday afternoon, Lynne Goldfarb began the last day of the semester for her master’s level humanities class in the University of Southern California’s education program. This was not a typical USC classroom; Goldfarb’s class is held weekly online, with just two students logging on from Los Angeles and Phoenix, Ariz.

“Today we’re going to look at how the two of you, in your own individual ways, in your own individual classes, have applied what you’ve learned in this class,” Goldfarb said, looking into her laptop’s camera.

With a few quick clicks, Goldfarb made of one her students the host of the online classroom, which would allow the student to share what was on her computer desktop with the class.

“This is sort of a game changer,” Goldfarb said, referring to the platform that USC uses for its online program, which allows students to see each other, share their computer screens, and chat live during class. She says that one of the benefits of the online program is the ability for students from across the country to share experiences and strategies with each other. “It’s a cross- pollination of sorts,” she said.

The online class is the product of USC’s evolving college of education, and a distant byproduct of the 1998 reforms. With fewer students enrolling in schools of education, an increasing number of traditional programs have started online components to draw in students who may find distance learning more convenient. The programs with the biggest enrollment numbers in California are now institutions with extensive online offerings, according to federal data.

But Karen Gallagher, dean of USC’s Rossier School of Education,  says that although the online program has high enrollment rates, there’s no data to show if the teachers trained online are better—or worse—than those trained in brick-and-mortar classrooms.

Both education schools and would-be reformers of teacher training have also embraced the idea of reaching out to a new population of potential teachers, because critics of teacher preparation programs say their biggest problem may be the kinds of people they recruit to become teachers in the first place. For years, colleges of education have battled reputations of attracting students with low test scores and grade point averages.

According to a 2011 College Board report, SAT takers planning to major in education scored an average of 480 in reading—above some, but below most disciplines, including law, engineering, and psychology. And among teacher preparation programs, admissions requirements vary greatly.

California requires a minimum score on the entrance exam students must take before they enroll in any teacher preparation program, but it’s extremely low. The cut score on the California Basic Educational Skills Test is 123 out of a top score of 240, or 51 percent— a percentage that would be considered a failing grade in most classrooms.

The test is split into three sections, which can be retaken as many times as needed, and scores from individual sections can be cobbled together to make a passing score. “If you are an intelligent ninth grader, you can probably complete it with very little problem,” said Kretschmer, the Northridge professor.

And while some schools have chosen to raise the cut scores or GPA required for admission, not everyone agrees that tougher admission requirements will result in a better teacher. “People say that’s a no brainer; you’re going to get better teachers if you increase the GPA,” said Spagna, dean at Northridge. “I would say that’s not a no brainer.” Spagna said some traits, such as having a cultural connection with students, may also have a positive effect on a person’s ability to be a good teacher.

At Northridge, students say the many requirements needed to graduate, and the packed programs that often require long days of student teaching followed by evening classes, have served them well.

Austin Trujillo quit his job in entertainment to enter the program, and says the program takes dedication and self-discipline—and that he is more confident about his job prospects than if he had chosen a newer, alternative program. “If you’re in competition and you have a degree from Northridge’s teaching credential program versus someone with an online degree, I think they’re going to assume you have a better-hands on experience,” he said. “It will be more respectable.”

Nancy Prosenjak’s class is filled with others like him, who were attracted to the program because they say it has a strong reputation among area principals. For these future teachers, it was all the data they needed to judge whether the program was working or not.