Fixing what’s *really* wrong with education research
Progress depends on connecting the dots between teaching and learning — not throwing more spaghetti at the wall.
By Ashley Jochim & Alexander Kurz, Senior Fellows, Center on Reinventing Public Education
Kelsey Piper hit a sore spot when she declared (via an essay in The Argument) that “Education research is weak and sloppy.” —
Concerns about the quality of education research aren’t new, but Piper’s piece arrived as the field continues to fight for its life after last year’s cuts to the Institute of Education Sciences. Mass layoffs and canceled projects have forced education researchers to do something they don’t often have to do: justify their work to an increasingly skeptical public.
Given this context, it’s not surprising that Piper’s piece didn’t land well among those who work in the field she set out to critique. Morgan Polikoff weighed in, calling Piper’s piece “ill-informed” and a “mess.”
We agree with Polikoff that much of what Piper suggests as solutions to the problems in education research are either ill-advised or already occurring.
But it would be a mistake to dismiss the problem she identifies.
It would be a mistake to dismiss the problem she identifies.
That problem–that education research lacks the rigor and impact found in other domains of science — is more serious than Polikoff’s response suggests, and it won’t be addressed with incremental improvements.
Yes, education research is more rigorous than in the past, thanks to the so-called “credibility revolution” that swept the social sciences. That revolution strengthened the standards for and tools researchers use to make inferences about what does and doesn’t “work” in education. It also led to many more applied economists contributing to educational research.
But this “rigor” is more superficial than it appears. Scientific inquiry is rooted in building explanations of the world as it exists, not just measuring the outcomes of our guesses. In medicine–a field that shares some features with education but more consistently delivers actual results — these explanations center on how biology and chemistry shape the course of disease.
This bedrock knowledge is used to create new, more effective treatments. The FDA requires drug companies and device manufacturers to show “proof of concept” before moving on to human clinical trials. Notably, this “preclinical research” largely occurs in the lab, where researchers use tightly controlled experiments to evaluate mechanisms of action and potential for toxicity.
This is not how it works in education. A “proof of concept test” in education would require interventions to be rooted in what we know about how teaching produces learning. The body of work is housed in the developmental and cognitive sciences, education’s closest approximation to medicine’s laboratory. But these fields are largely peripheral to the work in education and do not feature prominently in the field’s leading journals or in the grantmaking activities of IES.
The decade-long push for teacher evaluation reform provides a particularly noteworthy example of how this void has distorted policy and undermined scientific progress. This high-profile initiative didn’t make use of scientific knowledge on how the work of teaching produces learning. Instead, it treated effective teaching as a fixed trait of individuals–as if “good teaching” were a genetic lottery rather than a function of a well-designed instructional system. That theory, developed by economists, ignored many well-known constraints on good teaching, including inadequate preparation and poor instructional materials.
Amidst the debate over teacher evaluation, E.D. Hirsch declared, “The real problem is idea quality, not teacher quality.” We agree.
Science has important roles to play in addressing the idea problem, but it cannot hope to do so while treating students and classrooms as theoretical black boxes. We cannot think of another wing of science where interventions are based on theories that ignore the basic anatomy of the subject; we shouldn’t stand for it in education.
It may seem far-fetched that we could replicate medicine’s commitment to basic science, but the groundwork and bedrock knowledge already exist. Cognitive science teaches us that learning is a function of memory encoding (if you can’t remember, you haven’t learned) and that we can strengthen this encoding through specific instructional practices that reduce cognitive load (the extraneous material that doesn’t need to be learned), use carefully constructed memory retrieval tasks, and that require students to demonstrate their learning via regular tests.
This knowledge could be used to build interventions powerful enough to make a real difference. A developer of an AI-enabled tutor, for example, could ensure their instructional program incorporates instructional principles that already have proof of efficacy (for additional examples, see here). Does this guarantee that the product will “work”? Of course not. But at least we won’t be throwing spaghetti at the wall anymore.
The next iteration of IES could have a hand in supporting better integration between basic and applied research by investing in the basic research that applied researchers need and by requiring applied researchers to demonstrate that their interventions are rooted in that science.
This approach is already in place for the agency’s special education programs and should be scaled up to other funding priorities. Doctoral programs also have a role to play, and should ensure their trainees understand how students learn and how good teaching can support that outcome. These moves would bring education closer to medicine, where science has stronger results and—importantly—more public support.
Ashley Jochim & Alexander Kurz are both Senior Fellows at the Center on Reinventing Public Education. You can find their work here and follow them there.
Previously from The Grade