The more time I spend hearing “research-informed educational advice,” the more I worry about the enticing words “research-informed.”
Many MANY people toss around the phrase “research says…”; all too often, even a brief investigation suggests that research really doesn’t say that.
For this reason, I find myself slower to get excited about new “research-based” teaching ideas than many of my colleagues…even colleagues whom I admire, respect, and generally trust.
For instance: lots of scholars are investigating the field of embodied cognition and — more specifically — of using gestures to promote learning.
I’m certainly open to the idea that combining gestures with words and visuals will improve learning. And: I want to know A LOT more about the specifics of this idea:
Who is making these gestures? Teachers? Students? Actors in videos?
What kind of gestures are they? “Deictic” or”iconic”? Rehearsed or improvised?
Does the strategy work well in all disciplines/grades/cultures?
And so forth.
I’d also love to see some straightforwardly convincing research to support the answers to those questions.
So, for instance, I wrote a post about students using gestures to learn about Brownian motion. While the outline of the study made sense to me, it…
… didn’t have a control group,
… chose a topic easily translated into gestures, and
… measured “learning” 2 days later. (Does 2 days count as learning?)
While I’m glad I read the study, and appreciate some of its nuances, I don’t think it’s a slam dunk.
At the same time, I should turn some of my skeptical energy towards myself.
In other words: given all of my doubts, I should also be ready to doubt my own doubts. Maybe the wisdom of the crowd should outweigh my own habitual caution here. Maybe I’m so invested in my skeptic’s persona that I’m subconsciously unwilling to be persuaded…
Enter the Steelman
Because I doubt my doubts, I’m always on the lookout for EXCELLENT research contradicting my outlier point of view. I genuinely WANT to have my errors pointed out to me.
For that reason, I was delighted to find a highly touted study about teaching physics with embodied cognition.
My source here — published by the Educational Endowment Foundation — looks for the very best evidence supporting all sorts of cognitive science-based teaching advice: interleaving, retrieval practice, schemas, and so forth.
Of the 26 studies they found looking at embodied cognition, one stood out for its excellence. (In their rating system, it’s the only only one they rated “high priority.”) If the EEF, and all the wise scholars behind this report, find this study persuasive, it’s likely to be among the best research I can find.
In other words: I’m not analyzing a straw man here. This study is the “steelman.”
Playground Physics
The idea behind this study sounds both sensible and fun. Many of the abstract concepts studied in physics class are acted out quite concretely — that is, they are EMBODIED — when our children get to the playground.
If we could connect abstract classroom physics with embodied playground phyics, that approach could be really helpful.
This study begins with a good idea…and an ENORMOUS sample size. Over 3400 (!) students were in the initial sample; after (unusually high) attrition, that number dropped to about 1300 — roughly 800 in the “playground physics” group, and 500 in the control group.
The researchers wanted to see if the students in the playground group would a) learn more physics, b) feel more engaged, and c) feel more motivated — all compared to the control group.
The special “playground physics” program begins with a training session for the teachers, and includes curricular materials.
Crucially, playground physics also includes a phone app that students use to analyze their own motion:
“Using the app, users record videos of themselves and their friends engaging in physical play, and the app generates graphs of distance traveled, speed, direction, and kinetic and potential energy. As users watch the video, they see graphs of their movement unfolding. Users can pause to examine where they are moving fastest or slowest, where a force is pushing or pulling, and where their kinetic and potential energies are at their highest and lowest points. This is intended to support conversations grounded in the children’s physical experience”
Honestly, the whole experience sound really interesting!
Persistent Doubts
Although I tried to find a Steelman Study to support the case for Team Embodied Cognition, I’m still not persuaded.
I have two substantial concerns:
First:
This study does not measure the benefits of embodied cognition for learning physics.
Instead, it measures the benefits of embodied cognition PLUS cool tech gadgetry for learning physics. In fact, the study is published in a journal that focuses on technology in education.
Yes, the students learned more — but the extra learning could have come from the app (so much fun with video!) or from the embodied cognition (moving is so cool!) or both. We just don’t know.
I am not the only person pointing out this concern. The study’s authors say several times that they don’t know what the “mechanism” is that created additional learning. In other words: they do not claim that the embodiment matter more than the tech — or that it mattered at all. They don’t know.
To be persuaded by research into the use of gestures, I want to see a study that singles out the gestures; it should — in the lingo of research — “isolate the variable.” This one doesn’t.
Second:
When we compare two groups, we want them to be close enough to each other to be good proxies for each other. I’m not sure we can say that for this study.
A) The teachers of Playground Physics received extra PD; the teachers in the control group didn’t. Did the PD itself make the difference? We don’t know.
B) The study used a “business-as-usual control group.” That is: control group teachers just did what they always did. Teachers and students in the Playground Physics group got a Shiny New Thing. Was it the novelty that made the difference? We don’t know.
C) The Playground Physics group spent 15.5 hours studying physics; the control group spent 13.2 hours. The study’s authors write that this difference isn’t “statistically significant.” But — as a classroom teacher — I’m thinking two hours and fifteen minutes of additional practice would be significant, even if it isn’t “significant.” *
Because the study doesn’t isolate the variable (that’s the first concern) and the two groups don’t sufficiently resemble each other (that’s the second concern), I’m still stuck thinking: “this study doesn’t persuade me that embodied cognition is a thing.”
And — as you recall — I looked at this study because a respected group said it’s the best one they found.
TL;DR
I’m still looking for the study that makes the Embodied Cognition approach to teaching persuasive enough for me to recommend it to others.
I haven’t found it yet…but I haven’t given up hope.
By the way: if you know of such a study, please send it my way!
* I spoke with a stats-whisperer friend, who agrees with me that this simply isn’t a reasonable claim.
Margolin, J., Ba, H., Friedman, L. B., Swanlund, A., Dhillon, S., & Liu, F. (2021). Examining the impact of a play-based middle school physics program. Journal of Research on Technology in Education, 53(2), 125-139.
