The Hechinger Report is a national nonprofit newsroom that reports on one topic: education. Sign up for our weekly newsletters to get stories like this delivered directly to your inbox.

Locations of charter middle schools in Boston. Source: “Can Successful Schools Replicate? Scaling Up Boston’s Charter School Sector,” Sarah Cohodes, Elizabeth Setren, and Christopher R. Walters, NBER Working Paper No. 25796 May 2019

Research sometimes shows that charter schools are better at raising student achievement than traditional public schools. But many charter schools get about the same results and sometimes charter schools do worse. Since charter schools are funded by public tax dollars, it’s important to have a way to decide which ones to open.

A new Boston study suggests a possible path forward: allow only charter schools that can prove they are better than traditional public schools to expand. That might sound like common sense, but many states and districts don’t have such stringent standards. The study found that, at least in Boston, proven charter schools continued to produce outsized test score gains for students even as they added campuses and served more students.

“We see something that worked,” said Sarah Cohodes, a professor at Teachers College, Columbia University, and one of the study’s three authors. “Authorizing charter schools is supposed to be a rigorous process but different authorizers have different philosophies about the best way to do that. To our knowledge, Massachusetts is the only state that has this proven provider policy. This is a very promising path.” (The Hechinger Report is also based at Teachers College but is an independently run news organization.)

The study by Cohodes and two other academic researchers tracked what happened in Boston after 2010, when the state allowed more charter schools to open. The charter sector doubled, enrolling more than 30 percent of the city’s sixth graders, for example. But the state was very choosy about which new schools it permitted, approving only those charters with a proven track record of raising academic achievement. In the case of Boston, those schools subscribed to a “no excuses” philosophy. The schools tended to combine strict discipline with high expectations, increased time in school, frequent testing and tutoring along with standardized practices for hiring and training teachers. They largely enrolled low-income students of color.

The researchers focused on middle schools, studying three charter networks that already had schools in the city, Roxbury Preparatory, Brooke Charter Schools and Excel Academy Charter Schools, along with two newcomers, KIPP and UP Academy, which won approval based on their track records in other cities.  In all, seven new schools opened, doubling the number of charter middle schools in the city.

Related: Nearly 750 charter schools are whiter than the nearby district schools

The new charters were popular with Boston families; there were many more applicants than seats in the schools. Researchers compared test score gains for students who won a seat in admissions lotteries with those who didn’t and ended up attending a traditional public school. The researchers found that students’ test score improvements in the new schools were just as high as in the original group of high-quality charter schools. Charter school test gains continued to outpace those in traditional public schools and the charter school sector as a whole became even more effective, as measured by test scores.

That’s impressive because it’s rare in education to be able to take programs that work with a small group of students and expand them. Often, the tightly controlled conditions in a successful trial diminish with more students and teachers. It can be difficult to train larger groups of teachers properly and get them to follow intricate steps; some may not be enthusiastic about the approach or pedagogy.  Most things don’t “scale” well, as the business world would phrase it, because implementation is hard.

The study, “Can Successful Schools Replicate? Scaling Up Boston’s Charter School Sector,” is still a working paper, meaning it hasn’t yet been published in a peer-reviewed journal. But it was circulated by the National Bureau of Economic Research in May 2019. In addition to Cohodes, it was written by Elizabeth Setren at Tufts University  and Christopher Walters at the University of California, Berkeley.

Beyond documenting the effectiveness of the charter school expansion, the authors also wanted to understand why the “no excuses” charter schools were successful in reproducing strong results. They homed in on teaching because they noticed something unusual:  new teachers tended to be quite effective, as measured by student achievement increases, in their first years of teaching at the Boston charter schools. Typically, rookies improve dramatically in their first years of teaching and are much better after five years on the job than when they first started. The charter school teachers were learning on the job as they gained experience, too. But they were quite good immediately in their careers and didn’t have such steep learning curves afterwards.

The researchers also noted that teacher quality at the charter schools didn’t vary much. To be sure, some teachers were better than others; that is, they helped students’ test scores rise even more. But these differences weren’t so big. At traditional public schools, by contrast, there’s a huge range of teacher quality from very effective to very ineffective.

Related: Better adult outcomes (not test scores) for charter school students, Florida research shows

That led the authors to speculate that the standardization of hiring, training and teaching practices at these “no excuses” charter schools is driving the results. Much like at fast-food restaurants, there’s a lot that each charter school network does identically at every campus. The researchers learned through interviews that the curriculum at these schools was largely set by school leaders or collectively planned by experienced teachers. New teachers didn’t write their lessons from scratch. In Boston, experienced teachers from the parent campuses spent a lot of time mentoring teachers in the expansion schools. During the hiring process, network leaders sought new teachers who believed in the school model before they started.

“Standardized practices alone are not enough,” said Cohodes, “because you could have a consistent curriculum that isn’t a good curriculum. It has to be replicating something that has been deemed effective.”

Elsewhere in Massachusetts, the state has authorized many charter schools that aren’t so effective. Outside of Boston, the “no excuses” model isn’t used much. Charter schools do no better than or even worse than neighboring traditional public schools. However, districts outside of Boston haven’t bumped up against caps on the number of charter schools, Cohodes explained. The 2010 “proven provider” requirement only applies to districts that want to expand the number of charters beyond those limits. Unproven models are still allowed to open until then.

Related: More than 6,000 charter schools now operate in the United States

Across the nation, the “no excuses” model is falling out of favor with critics questioning the harsh disciplinary practices and finding evidence that they’re excluding students with disabilities. And it’s not the only charter school model to show positive results nationally. On the opposite end of the spectrum is High Tech High, a charter school network in California, which boasts both high test scores and college attendance rates. It has a more flexible approach that gives some autonomy to teachers and encourages students to come up with their own projects.

One of the biggest criticisms of charter schools is that they weed out difficult students. The researchers developed two ways to see if this was a factor in their study. First, they continued to track students who left charter schools, and classified them as charter students in their school effectiveness calculations. Their subsequent public-school test scores were attributed to the charter sector. But the charter sector’s gains remained strong and didn’t vanish. Second, the researchers counted how many students left their schools two years after the lottery and found no evidence that students were leaving charter schools at higher rates. (They found that all the students who want to attend charter schools and enter a lottery are a mobile bunch and are more likely to leave their school than the general population. Students who lost the lottery and didn’t get a charter school seat initially were even more likely to leave their assigned public school than the kids who won a a charter school seat.)

Deciding which charter schools should be allowed to receive public funds and educate children is tough. This proven provider study points to a tension between using the charter school sector as a laboratory of innovation while insisting that it must outperform traditional public schools. If we insist on the latter, that might tip the scales to large charter chains that emphasize testing, ignoring other things that we value.

This story about charter schools in Boston was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The Hechinger Report provides in-depth, fact-based, unbiased reporting on education that is free to all readers. But that doesn't mean it's free to produce. Our work keeps educators and the public informed about pressing issues at schools and on campuses throughout the country. We tell the whole story, even when the details are inconvenient. Help us keep doing that.

Join us today.

Letters to the Editor

At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.

By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.

Your email address will not be published. Required fields are marked *