As I wrote last week, thinkers in edu-world often make strong claims at the expense of nuanced ones.
For example:
- “A growth mindset undergirds all learning” vs. “growth mindset is an obvious boondoggle.”
- “AI will transform education for the better” vs. “AI will make people dumber and schools worse.”
- “Be the sage on that stage!” vs “get off the stage to guide from the side!”
The list goes on (and gets angrier).
When researchers start digging into specifics, however, the daily experience of teaching and learning gets mightily complicated, and mighty fast.
All those strong claims start to look…well…too strong for their own good.
One extraordinary example of “digging into the specifics” can be found in Graham Nuthall’s The Hidden Lives of Learners. Nuthall put cameras and mics on students in New Zealand classrooms, and arrived at all sorts of astonishing conclusions.
Another recent study looks quite specifically — no, really specifically — at 4 teachers. The goal: to understand what part of their work helped students learn.
Here’s the story.
Time to Review
A group of scholars, led by Dr. Janet Metcalfe, wondered if students learned more from teachers’ responses to their mistakes than from teachers’ direct instruction. (You can learn more about the study here.)
A few important points merit attention right away.
First: the classroom sessions I’m about to describe are REVIEW sessions. The students have ALREADY learned the math covered in these lessons; the teachers are helping them review in preparation for a high stakes exam.
In other words: this study does not focus on initial instruction. It focuses on subsequent review.
Second: the students involved VOLUNTEERED to take part. They are, presumably, atypically motivated to learn math.
Keep these points in mind as you think about applying the ideas described below.
In this study, 4 teachers helped 175 8th grade students prepare for upcoming state math exams.
For half of the students, the teachers taught 8 lessons (“explicit instruction”) reviewing core math concepts that would be on that exam.
For the other half, the teachers responded to the misakes that students made on practice tests. That is: during 4 sessions, students took 45 minute math tests. And after each of those sessions, the teachers
“were instructed […] to focus on the students’ errors and to do whatever they deemed appropriate to ensure that the issues underlying the errors would not reoccur and that the students would learn from their errors.”
So, which review approach proved more helpful — the explicit instruction, or the learn-from-mistakes instruction? And, why?
An Envelope, and LOTS of Questions…
The answer to that first question — which kind of review proved most helpful? — is easy to answer.
Students in both groups learned math; they did better on the post-test than the pre-test.
The students in the “learn-from-mistakes” group learned more.
This straightforward finding leads to obvious questions. And — alas — those obvious questions are VERY tricky to answer.
For instance, “how much more did the students in the learn-from-mistakes group learn?” That’s a reasonable question. The answer takes some careful parsing.
Roughly speaking, students in the explicit instruction group increased their scores about 2% per hour of instruction.
For those in the learn-from-mistakes group, the answer depended on the teacher.
The least successful teacher helped students in this group improve 2% per hour of instruction. The most successful teacher helped students improve 5% per hour of instruction.
Of course, that last paragraph prompts another reasonable question: what was different about those two teachers? Why did one teacher benefit his/her students more than twice as much as their colleague?
Let the Sleuthing Commence…
The study’s authors spend a great deal of time — and crunch a great many equations — to answer that question.
For instance:
Maybe the teacher whose students learned more (let’s call her Teacher M) is just a better teacher than the one whose students learned less (Teacher L).
As the researchers point out, that explanation doesn’t make much sense. After all, in their explicit instruction sessions, both Teacher M and Teacher L helped their students equally.
(By the way: to simplify this blog post, I’m leaving out the two other teachers for now.)
Okay, maybe Teacher M did a better job of focusing on students’ mistakes, whereas Teacher L spent too much time focusing on questions that students got right.
Nope. This study includes quite an eye-watering graph to show that they both focused about the same on students’ mistakes.
As the researchers write: “all of the teachers taught to the errors of their students, and … the extent to which they did so did not predict student learning.”
So, what was the secret sauce?
The Perfect Combination
After a few more false leads, the study focuses on two moment-by-moment variables: the teachers’ focus, and the kind of interaction with the student.
Focus: did the teachers
“[dwell] upon how to solve the problem correctly,” or
“[delve] into the nature of the errors – why the students had made them, what the difficulty in the logic was, and/or how to recognize and circumvent such mistakes in the future”?
Kind of interaction: did the teachers explain/lecture, or did they discuss/interact?
With this pair of questions, at last, the study struck gold.
Teacher L — whose students learned relatively little — focused almost all her time on “how to solve the problem correctly.” While pursuing that goal, she divided her time equally between lecture and discussion.
Teacher M — whose students improved more quickly — spent almost all her time in discussion, with almost no time in lecture. While in this interactive mode, she divided her time more-or-less equally between solving problems and understanding the nature of the mistake.
This final insight allows us to make this claim:
Highly motivated 8th grade math students,
reviewing in preparation for a high-stakes exam,
learn less from explicit instruction and more from making and reviewing their mistakes,
as long as the teacher keeps those review sessions interactive,
and equally focused on “getting the answer right” and “understanding the nature of the mistake.’
Notice, by the way, all the nuance in this statement.
To emphasize just one point here: this study does NOT argue that “learning from mistakes” is better than “direct instruction” in all circumstances.
It argues that students learn more from mistakes when reviewing, as long as the teacher follows a very particular formula.
A Final Note
Heated battles in this field often get hung up on specific labels.
As I’ve written before, we do a LOT of arguing about benefits of “desirable difficulty” vs. “productive struggle” — an odd set of arguments, given that both phrases seem to mean the same thing.
This study was co-authored by (among other scholars) Robert Bjork — who helped coin the phrase “desirable difficulty.” For that reason, you might be surprised to learn that this study touts the benefits of “productive struggle.”
That is: the students took a test, they made mistakes, they wrestled with those mistakes, and they learned more. Their struggle (trying to understand what they did wrong) was productive (they improved on their test scores — and probably their understanding of math).
Of course, I could just as easily describe that process as “desirable difficulty.” The difficulties these students faced here — the test, the mistakes, the analysis — turned out to be beneficial — that is, “desirable.”
My own view is: don’t get hung up on the label. The question is: are the students both thinking harder and ultimately succeeding? If “yes” and “yes,” then this teaching approach will benefit students.
Metcalfe, J., Xu, J., Vuorre, M., Siegler, R., Wiliam, D., & Bjork, R. A. (2024). Learning from errors versus explicit instruction in preparation for a test that counts. British Journal of Educational Psychology.
