If you want a glimpse into cutting-edge research on artificial intelligence in education, look to Jing Liu, director of the Center for Educational Data Science and Innovation at the University of Maryland. Two studies released in October offer a striking and enlightening contrast. One hinted that a little AI-powered math practice might help struggling middle schoolers improve their skills. The other found that an AI assistant undermined college students’ engagement and hurt their grades.
Liu wanted the studies to be released together, he told me, to show that AI in education is neither “all good” nor “all bad.” The field is still figuring out how to design tools that improve learning over time. And that means studying not just the technology, but also how students respond to it — including in ways its creators never intended.
In the first experiment, a math practice website called Goblins was rolled out to 6,000 middle schoolers in Deer Valley, Arizona, during the 2025-26 school year. Teachers assigned problems to their students. An AI tutor offered step-by-step help through text chatting, drawing on a virtual whiteboard and even spoken conversation. Teachers were encouraged to use it with struggling students, though some assigned it to their entire classes and others skipped it altogether.
Students who solved at least two problems a week — more than 50 over the school year — scored significantly higher on a year-end standardized assessment in math than similar students who used Goblins less or not at all. Accelerate, an independent, nonprofit research organization that commissioned the study, equated the statistical difference of 0.11 standard deviations to about three additional months of learning.
But hold off on the celebratory bot music. The study cannot prove that Goblins caused the gains. Students who used it more might simply have been more motivated to practice math and might well have tested better in math with or without Goblins. Accelerate hopes to commission a randomized controlled trial to settle the matter.
Still, one result from this study is worth highlighting: Half the 6,000 students used Goblins to tackle at least two problems a week, the dosage associated with higher math scores.
That’s noteworthy because education technology is plagued by something that education researchers call the “five-percent problem.” A tool may work beautifully for students who use it as directed, but as a rule of thumb, only about 5 percent actually do. The other 95 percent either don’t log in at home or sit idly through the minutes of scheduled screentime in class. Studies on Khan Academy, IXL and other platforms have well documented this challenge of getting children to use their instructional software consistently.
Goblins appeared to reach some students who are often hard to engage, including English learners and students with disabilities. Liu suspects that’s because students could talk aloud to a cartoon avatar of their teacher, which could talk back. Moreover, the AI tutor could speak and understand Spanish and six other languages.
“It was like talking to your own teacher,” said Liu. “It made the students want to use it.”
Teachers’ control over assignments may have helped, too. Rather than surrendering the lesson to a computer, teachers decided which problems to assign to which students and had the option to upload their own problems. That echoes research findings from older education technology. Another middle school math platform, ASSISTments, also produced strong results using relatively small amounts of teacher-assigned practice with instant feedback for students.
Still, I don’t want to overstate how engaging or appealing Goblins was. Only 6 percent of the students in this experiment used it for at least half an hour a week, which is a more common dosage recommendation in ed tech and for which Goblins’ usage mirrors the familiar five-percent problem.
When AI access backfires
Liu’s other experiment tested an AI assistant developed at the University of Maryland. Before deploying it across the university to all 40,000 students, Provost Jennifer King Rice asked Liu to study it first.
During the fall semester of 2025, Liu randomly assigned 15 of 30 professors to add the AI assistant to their classes. The other 15 taught the same courses, without AI. Subjects ranged from English and math to biology and social sciences. For example, two professors taught separate sections of the same biology course, one with the AI assistant and one without.
Professors supplied the university’s AI chatbot with syllabuses, readings, assignments and other course materials. During the semester, students could ask about assignment instructions and deadlines or get help understanding concepts. The assistant was available around the clock through Canvas, the university’s online learning management system.
By the end of the year, the 1,100 students in courses with the AI assistant had lower grades — reduced by an average of 0.11 points on a four-point scale, roughly the difference between a 3.0 (a solid B) and a 2.9 (a low B) – compared with the 1,200 students in the control group. They also interacted less with course materials (such as making a comment on the class’s online discussion board) and were less likely to attend office hours or meet with instructors and peers. The negative effects were more pronounced for first-generation students.
“AI tools could exacerbate inequities in learning outcomes,” Rice told a University of Maryland publication. “That is particularly troubling to me.”
Only 15 percent of students in the AI-enhanced classes ever used the assistant. Those who did mostly asked it for answers rather than taking advantage of its tutoring features.
How could a tool used by so few students have such negative effects?
Liu and Rice suspect that the AI assistant’s constant presence in the course website sent an unintended message: AI use is encouraged. Students may have taken that as permission to use ChatGPT, Claude or Gemini outside the university’s platform to complete assignments. The researchers cannot establish whether that happened. But if AI did the work for students, it may have left them learning less.
The lesson isn’t that colleges should avoid AI, says Liu. It’s that even a carefully designed tool can have consequences beyond its intended use. And Maryland deserves credit for testing the assistant before a university-wide rollout and for making the troubling results public for the whole field to learn from.
Contact staff writer Jill Barshay at 212-678-3595, jillbarshay.35 on Signal, or barshay@hechingerreport.org.
This story about AI tutor research was produced by The Hechinger Report, a nonprofit, independent news organization that covers education. Sign up for Proof Points and other Hechinger newsletters.
More Proof Points
Republish our articles for free, online or in print, under a Creative Commons license.
