The hottest trends this fall on campus? Denim jeggings. Nerd glasses. Studded backpacks. Instagram.
Hailed by politicians and journalists as the affordable future of higher education, the MOOC is neither fashion accessory nor smartphone app. It’s a massive open online course—a college class available for free online to tens of thousands of people at once.
And it’s getting crazy buzz. The Economist asked this summer if the MOOC portends “the fall of the ivory tower.” Administrators from dozens of top universities and colleges are scrambling to get on the MOOC train. No wonder; a third of them say they think that MOOCs will eventually replace residential campuses. Students are jumping aboard, too. Enrollments are ballooning by the hundreds of thousands each semester.
There’s just one hitch: Amid all this rush, no one really knows yet how much people learn in a MOOC. What research does exist shows that the success rate of online education, in general, is poor. And one high-profile experiment with MOOC-style teaching in particular has ended in disappointment.
“At this point, there’s just no way to really know whether they’re effective or not,” said Shanna Jaggars, assistant director of the Community College Research Center at Teachers College, Columbia University, which has produced some of the most recent scholarship about online education.
“My approach would be to tread carefully,” Jaggars said. “You need to try things in very small steps with small groups of students and carefully look at where students are falling behind or getting confused or losing their motivation. Until you’re getting very good outcomes with vulnerable students, I wouldn’t want to widen it beyond a small group.”
It may be too late for that. Enrollment in online college courses of all kinds increased by 29 percent to more than 6.7 million between 2010 and last year, the latest period for which the fast-changing figures are available, according to the Babson Survey Research Group. And this is happening at a time when the number of students in conventional universities and colleges has started to decline.
MOOCs alone—as opposed to other kinds of online classes, including those with limited enrollment and for which tuition is charged—are growing so quickly, it’s impossible to know how many people take them. Barely a year and a half after its debut, Coursera, a startup launched by Stanford faculty, reports that about 4.4 million students have signed up for its MOOCS. The MIT-Harvard MOOC collaboration called edX, which premiered just four months later, boasts more than a million.
Then again, about 90 percent of people who register for MOOCs fail to complete them, most providers acknowledge. Advocates say that’s because there are no admissions requirements and the courses are free; they compare the idea to borrowing a book from the library and browsing it casually or returning it unread.
A survey of students by the market-research company Qualtrics and the education technology provider Instructure seems to confirm this. Seventy-five percent said the main reason they signed up for a MOOC was that it didn’t cost them anything, while 29 percent of those who dropped out said they got too busy to continue, and 20 percent said they lost interest.
Two-thirds of those students said they would be more likely to complete a MOOC if they could get college credit or a certificate of completion for it, something that is still not widely available. Until it is, said Jaggars, it will be hard to measure the effectiveness of MOOCs—a Catch-22, since without knowing their effectiveness, it’s unlikely colleges will give academic credit for them.
To study what happens when students do get credit for online courses, Teachers College looked at online courses at community colleges in Virginia and Washington State that were not MOOCs—since tuition was charged and credit given—but were like them. The results were not encouraging. Thirty-two percent of the students in online courses in Virginia quit before finishing, compared with 19 percent of classmates in conventional classrooms. The numbers in Washington State were 18 percent versus 10 percent. Online students were also less likely to get at least a C, less likely to return for the subsequent semester, and ultimately less likely to graduate.
In July, San Jose State University suspended an experiment with offering MOOCs for credit after only half of credit-seeking students who took the online courses passed, compared to three-quarters of those who took the traditional versions. In one of the three pilot classes, which were offered during the spring, fewer than 30 percent of the online students passed. And while the university and its partners hailed an apparently dramatic improvement in results in the summer semester, a closer look showed that the summer students showed up considerably better prepared. More than half already had at least a bachelor’s degree, compared to none of the students who took online courses in the spring—and even then, more of the summer registrants dropped out.
“In general, students don’t do as well in online courses as they do in conventional courses,” said Jaggars. “A lot of that has to do with the engagement. There’s just less of it in online courses.”
That’s because, while their scale is unprecedented, the underlying way that MOOCs are taught isn’t really new, Jaggars and others said. It’s very, very old—a system in which huge numbers of students are lectured to by professors with whom they seldom, if ever, interact.
Even in MOOCs that encourage students to hold online conversations on discussion boards, a maximum of only about one in five ever bother, Stanford researchers found.
“I do take umbrage with so many journalists and people in the field talking about MOOCs as if they’re some brand-new innovation,” said Dan Leopard, a professor at Saint Mary’s College of California and author of a book about the history of technology in education. “There has to be some caution about how truly innovative it is.”
The people who offer the MOOCs themselves bristle with the idea advanced by journalists, politicians, and some university administrators that this new model will transform all of higher education.
“I really hope you’ll help to dispel that,” said Andrew Ho, a professor at the Harvard Graduate School of Education and research director for HarvardX.
“Everyone in the research field agrees that, for the particular purpose of replacing on-campus education, the evidence [for MOOCs] is ambiguous at best,” Ho said. “Far more research is needed. And we’re conducting some of it. But we’re way out over our skis when it comes to that particular purpose of MOOCs.”
He said: “I am not aware of any large-scale rigorous experimental study that has evaluated Udacity-, Coursera- or edX-type MOOCs, now or planned for the near future, for what many legislators seem to be wanting to use MOOCs for. If that is a serious policy concern, then somebody needs to step up and fund the research.”
In spite of all this, 77 percent of academic leaders think online education is as good or better than face-to-face classes, and 69 percent say that it’s essential to their long-term strategy, the Babson group found (though the administrators also conceded that only 30 percent of their faculty agreed). Four in 10 said their schools plan to offer MOOCs within three years, according to a survey released in the spring by the IT company Enterasys. And legislators in several states are pushing to speed up the shift to MOOCs for college credit, which they see as a way to expand access to higher education while reducing costs.
So far, what’s in it for the universities and colleges seems to be not getting left out. It’s still unclear whether MOOCs can make enough money to cover their costs through such things as fees for proctored exams, for example, through which students would prove that they’ve mastered the material. Yet expectations remain high. Harvard and MIT are investing $60 million in edX, and Coursera has raised more than $65 million in private venture funding.
Ho said that’s a good investment, since the methods and technology used in MOOCs can be transferred to improve education in conventional classes.
“Comparable amounts of money have been spent for far less interesting data,” he said.
But some people want to slow things down, especially when it comes to using MOOCs to teach 18- to 22-year-olds.
“I find it extraordinary that this has become the topic of the moment,” said Donald Eastman, president of Eckerd College. “It is nonsense for any public or private university to pretend that online courses for undergraduate students provide quality education.”
Yet people seem to be racing to embrace MOOCs, said Franklin & Marshall College President Daniel Porterfield, “without any research about how to deal with issues, for example, of different levels of student preparedness, the limits of assessment that can be applied to 100,000-student classes, the possibilities of academic dishonesty occurring, and the impact of having 90 to 95 percent dropout rates.”
And John McCardell, vice chancellor of Sewanee: The University of the South, said deep-seated skepticism about MOOCs is the reason some of the very universities that offer them won’t give academic credit for them. Only 10 MOOCs—one from edX, five from Coursera, and one from another MOOC provider called Udacity—have been recommended for credit by the American Council on Education, and even that recommendation is advisory only and not necessarily binding on colleges and universities, which determine for themselves whether or not to confer the credit.
“That ought to tell you something,” McCardell said.
He quoted the poet Alexander Pope, whose poetry he took pains to point out that he had studied in an old-fashioned brick-and-mortar classroom years ago.
“Be not the first by whom the new are tried, nor yet the last to lay the old aside,” Pope wrote.
“That might be a useful thought to keep in mind,” McCardell said, “as the world seems to be rushing headlong to embrace this latest pedagogical fad.”
This story is an exclusive collaboration with TIME.com. Reproduction is not permitted.