Last week, I described my enduring concerns about “embodied cognition.” I’m not sure I understand the concept clearly: what exactly counts as “embodied cognition” — mindfulness? Direct instruction? (No, seriously, a well-known book on the subject says it does!)
And the “best research” supporting some of the claims doesn’t feel persuasive to me.
Could using gestures help learning? SURE. Have I found enough research for me to advocate for this strategy? Not yet…
This week, I wanted to test my doubts about project-based learning (universally acronymed as PBL). SURPRISE: I end up feeling kinda persuaded — at least in this one case.
Here’s the story…
Another Steelman
If I’m going to critique a teaching method, I want to be sure to avoid straw men. Neither you nor I learn anything if I point out the flaws in an obviously foolish study or approach. I’m going to learn something if and only if I take on the very best case.
Some thoughtful soul — I’m embarrased to say, I can’t remember who — recommended this PBL study to me.
Given the strength of that recommendation, I thought it worth a read — despite my PBL concerns.
What are those PBL concerns?
As is so often the case for me, I worry about working memory overload. If I ask my students to
- Film a scene from Hamlet, but re-imagine it in a new setting, or
- Build a model city that enacts 3 core principles of ecological design, or
- Write a new law that prevents a problem in our school’s community
I’m certainly giving them a rich cognitive task.
However, they almost certainly don’t have enough foundational knowledge to manage any of those tasks. Heck, graduate students in those fields struggle with such problems.
So, while I find the questing adventurousness of such tasks intriguing, my knowledge of working memory limitations tells me: ain’t gonna happen.
I should also confess: my experience assigning project-y work hasn’t gone well.
In brief: although “constructivist” approaches often sound appealing, my focus on basic cognitive capacities makes me extra skeptical.
(Important note: “constructivism” is an ENORMOUSLY broad category, and it’s inaccurate/unfair to lump so many pedagogies together into one ill-defined word.)
The Goals; The Problems
When I look at research, I’ve got a few desiderata:
One: The study should — as much as possible — isolate the variable. I can’t say that (to take a comic example) “chewing gum improves learning” if the participants both chewed gum and tap-danced.
Another one: the study should have a plausible control group. The question isn’t “did X improve learning?” but “did X improve learning compared to the plausible alternative Y?”
Yet another one: the researchers should try hard to measure what they claim. If I say “PBL helps students learn stuff,” I should have some reliable measurement of what they learned. If reseachers make up their own test…well…I worry that they’re (subconsciously) putting a thumb on the scale.
Because I’m a PBL doubter, I read this study with a keen eye on those topics. I’m used to finding such problems. For instance:
Isolate the variable: the study about “using gestures” actually used gestures AND cool tech stuff. I don’t believe claims about X if the students did both X and Y.
Plausible control group: again, the “using gestures” study compared teachers who got something (extra PD; extra curricular materials) with teachers who got nothing (no extra anything).
Measuring the claim: a study claiming that “handwriting helps students learn” didn’t measure learning. (I still can’t get over how many people are citing this study despite this extraordinary flaw.)
So, would this PBL study fall short of these standards?
To be clear — and fair — no study is perfect. Psychology is complicated; teaching is complicated; PEOPLE are complicated. So, I’m not asking that everything be perfect.
But I am asking that the study make a good-faith effort on most of those things.
Envelopes, Please
As a skeptic, I was pleasantly surprised by what I read. Two points stood out in particular:
First: unlike the “gesture” study, the PBL study made an impressive effort to treat teachers in both groups equally.
- Both groups — not just the PBL group — got extra PD time.
- Both groups — not just the PBL group — were told that classroom visits were a part of the program.
This kind of care is, in my experience, unusual. I was pleasantly surprised.
Second: the “measurement” sounds (largely) plausible. The researchers did NOT simply make up their own test of the science learning.
Instead, they used the Michigan State standardized test for both the PBL group and the control group. For time reasons, they didn’t use all the questions from that test — so they did have a change to put that thumb on the scale. But they had less of a chance than if they’d simply created their own test.
Now, don’t get me wrong. I do have some concerns. For instance:
- Although the teachers in both groups got special treatment, the students didn’t. That is: both groups of teachers got extra PD, but the students in the control group got “same old, same old.” The study would be more persuasive if they too got a new teaching approach.
- The teachers in both groups got extra stuff, but the teachers in the PBL group got MORE extra stuff. They got more (and more frequent) PD, and more curriculur support, and class visits. (For scheduling reasons, the promised class visits for the control group largely didn’t happen.)
- As noted above, the research team didn’t exactly use someone else’s measurement — although it seems they made a good-faith effort to do so.
In brief, I can quibble with the study — but I don’t think its flaws merit easy disqualification.
Final Verdict
The research team measured LOTS of variables, and scrupulously tallied scores for MANY important sub-groups and special circumstances.
If we take the headline number, they found an effect size of 0.277 (technically, “small”) for the amount of additional science knowledge that the students in the PBL group learned compared to the control group.
That is: PBL produced more learning, but not lots-n-lots. We can’t rule out the possibility that all that extra learning resulted from the “shiny new thing,” not from the PBL.
At the same time, my concerns about working memory overload were — at least in this one example — calmed. If this PBL program had overwhelmed WM for these 3rd graders, they wouldn’t have learned much at all; instead, they learned a bit more.
I still have lots of questions and concerns. But I’m heartened to see that — done right — this PBL program offers a potential pathway for further exploration.
Krajcik, J., Schneider, B., Miller, E. A., Chen, I. C., Bradford, L., Baker, Q., … & Peek-Brown, D. (2023). Assessing the effect of project-based learning on science learning in elementary schools. American Educational Research Journal, 60(1), 70-102.
