Print | Email |

Thinking we know what we test

By

An op-ed in The New York Times on August 20th,Testing What We Think We Know,” argued that many medical procedures are carried out in the United States despite a very thin evidence-base for their efficacy. It’s high time to invest more in research, the author wrote, to figure out first what actually works. The op-ed’s author, H. Gilbert Welch, is a professor of medicine at the Dartmouth Institute for Health Policy and Clinical Practice and a co-author of Overdiagnosed: Making People Sick in the Pursuit of Health (2011).

Welch’s op-ed about the field of medicine could just as easily have been about the field of education (but then, would The Times have published it?). The problems besetting both are strikingly similar. In that spirit, what follows is a riff on Welch’s op-ed, and it’ll likely make sense only if you first read “Testing What We Think We Know.”

By 2010, many politicians were recommending top-down accountability to healthy schools and rigorous assessments to determine the effectiveness of older teachers. Both interventions had become standard educational practice.

But in 2020, a randomized trial showed that top-down accountability caused more problems (more teaching to the test and cheating) than it solved (fewer bad schools and under-educated students). Then, in 2029, trials showed that top-down accountability led to many unnecessary tests and had a dubious effect on school improvement.

How would you have felt—after over two decades of following your elected official’s advice—to learn that high-quality randomized trials of these standard practices had only just been completed? And that they showed that both did more harm than good? Justifiably furious, I’d say. Because these practices affected millions of American schoolchildren, they are locked in a tight competition for the greatest educational error on record.

The problem goes far beyond these two. The truth is that for a large part of pedagogical practice, we don’t know what works. But we pay for it anyway. Our annual per capita K-12 educational expenditure is now over $11,000. Many countries pay half that—and enjoy similar, often better, outcomes. Isn’t it time to learn which practices, in fact, improve our educational system, and which ones don’t?

To find out, we need more education research. But not just any kind of education research. Education research is dominated by research on the new: new tests, new technologies, new disorders and new fads. But above all, it’s about new markets.

We don’t need to find more things to spend money on; we need to figure out what’s being done now that is not working. That’s why we have to start directing more money toward evaluating standard practices—all the tests and treatments that policymakers are already pushing.

There are many places to start. Value-added assessments are increasingly finding microscopic abnormalities in the teacher lounge called M.U.T.S., or Maybe Underperforming Teachers. Currently we treat them as if they were invasive cancers, with public shaming, firing and school closures. Some elected officials think this is necessary, others don’t. The question is relevant to more than 3.5 million teachers each year. Don’t you think we should know the answer?

Or how about this one: How should we screen for underperforming students? The usual approach, standardized testing, is simple and cheap. But more and more students and parents are opting out of public schools—over five million students attend private schools alone. And 1.5 million are home-schooled. Untold thousands go to virtual schools, where they learn at home in front of computers. These options are neither simple nor cheap. Which is better? We don’t know.

Let me be clear, answering questions like these is not easy. The Department of Education is in fact preparing to take on the question of whether underperforming youngsters can be made to perform like their peers. The trial, which will involve up to 50 million students, will last a decade and surely cost billions of dollars.

Research like this takes more than grant money. For starters, it takes a research infrastructure that monitors what standard practice is—data on what’s actually happening across the country. Because of PISA, we have a clear view for students aged 15, but it’s a lot cloudier for those under or over 15. Basic questions like how common illiteracy is and what testing is done to determine rates of illiteracy are unanswerable.

It also takes a research culture that promotes a healthy skepticism toward standard pedagogical practice. That requires teacher-researchers who know what standard practice is, have the imagination to question it and the skills to study it. These teachers need training that’s not yet part of any education school curriculum; they need mentoring by senior researchers; and they need some assurance that investigating accepted approaches can be a viable option, instead of career suicide.

We have to move quickly. The administrative demands of teaching, on one side, and the competition for school funding on the other, make it increasingly difficult for teachers to instruct students. They become isolated from standard practice, and their ability to study it diminishes. School leaders who are well positioned to study these issues are increasingly directed toward enhancing productivity—questions about how can we do this better, faster or more consistently—instead of questions about whether the practices are warranted in the first place.

Here’s a simple idea to turn this around: devote 1 percent of educational expenditures to evaluating what the other 99 percent is buying. Distribute the research dollars to match the instructional dollars. Figure out what works and what doesn’t. The Institute of Education Sciences (created as part of the Education Sciences Reform Act of 2002) is supposed to tackle questions of direct relevance to students and teachers and could take on this role, but its budget—less than 0.003 percent of total spending on education—is far from sufficient.

A call for more educational research might sound like pablum. Worse, coming from an educational researcher, it might sound like self-interest (cut me some slack, that’s another one of our standard practices). But I don’t need the money. The system does. Or if you prefer, we can continue to argue about who pays for what—without knowing what’s worth paying for.

Comments & Trackbacks (5) | Post a Comment

[…] Justin Snider has written an interesting article in the Hechinger Report. […]

Linda Johnson

Actually, we DO know a great deal about how to best educate a child, but for some strange reason (probably money) we are ignoring the data. We know that the following factors have a positive effect on learning:

Normal development in the womb (no drugs, alcohol, etc.);
Normal birth weight;
Protection of the young child (no child abuse, lead poisoning etc.);
Normal development in the first five years;
Superior vocabulary when entering kindergarten;
Number of books in the home;
College education for parents, especially the mother;
Two parents in home;
High-quality preschool;
School attended by many high-achieving children;
Amount of time parents read to children and children read to self;
Quality of the classroom teachers;
Desire of the student to excel;
Engagement of the student in the learning process;
Etc.

The research also tells us that American children not in poverty have the highest test scores in the world. We know that our schools in affluent neighborhoods can compete nicely with schools everywhere. In fact where I live (CA), parents from Korea and China send their children to Whitney High School , Oxford Academy and San Marino High School to study.

No, the problem is not a lack of research. The problem is this: What do we do with the impoverished child who was born with drugs in utero, who is malnourished and in poor psychological and physical health? How do we help the five-year-old who has a very poor vocabulary and seems unable to focus on instruction? What do we do about the child who comes to school only half the time?

We have known for a very long time what is required to educate a child, but we have neither the will nor the resources to apply this knowledge towards the education of other people’s children. All that we are doing now – merit pay, firing “bad” teachers, establishing common core, starting charters etc. – is a good example of “rearranging the deck chairs on the Titanic” and soon everyone will know. But will we do anything about it? My guess is yes, but it might be a very long wait. Basically we’ll have to wait until the cost of supporting poorly educated adults becomes so prohibitively expensive that taxpayers demand real answers.

In the meantime, journalists like Justin Snider are helping a great deal by spreading the truth about the fraudulent practices of the present “reform” movement. Thank you, Justin!

More research in education « Opinião

[…] research in education August 26, 2012 // 0 To find out, we need more education research. But not just any kind of education research. Education research is dominated by research on the new: new tests, […]

[…] Ravitch comenta em seu Blog 26/08/2012 que Justin Snider escreveu um artigo interessante no Relatório Hechinger. A autora se […]

[…] A writer riffs — with an edu-spin — on a Times op-ed that called for more research in health. (Hechinger) […]

Join the discussion. Your email is never published or shared.

Required
Required
CAPTCHA Image