Few educational innovations have gotten more hype than online learning, and few have a more checkered track record.
For every uplifting story we hear about a Khan Academy success, we get at least one story about massive drop-out rates for MOOCs.
You’d be forgiven for thinking that people are just better at learning face to face than from a computer.
And so a recent study deserves our attention. And — like all other research — it also merits our respectful skepticism.
Three Headlines, and a Warning
First Headline: middle schoolers learned A LOT more science from an interactive online curriculum than from a more traditional one.
Second Headline: learning gains seem impressive for students with learning disabilities, and for English language learners.
and
Third Headline: the program was tested with more than 2300 students and 71 teachers in 13 schools in two different states: Oregon and Georgia. That is: this study isn’t about 30 college students in a psychology lab. It’s about real students in real classrooms at opposite ends of the country.
And now, the Warnings: I’m going to give you reasons to be skeptical about all of this. (Well, at least the first two headlines.)
Interactive Online Science Teaching
Researchers developed four units aligned with national science standards. These units, each roughly 10-14 weeks long, covered such topics as Knowing My Body and Our Place in the Universe.
We should note that researchers brought several theoretical perspectives to these science units. They took care to make them culturally relevant to the learners. They designed each unit with project-based learning principals in mind.
And, they made sure that students could interact with the material in different ways. Middle schoolers could control the pace of the lesson, move images around on the screen, get feedback, and navigate among several sources of information.
Even from this brief description, you can see how this interactive online science lesson could be especially appealing — and potentially especially effective — for 7th and 8th graders.
(You can learn more about the curriculum at this website.)
Results: By The Numbers…
First, the online units helped students learn.
In the control group, where teachers taught as they typically had done, students improved 5.7% from pretest to posttest. In the treatment group, where teachers used the units described above, students improved 16.7%.
When the researchers looked at students with learning disabilities and at English language learners, the raw data also showed promising improvements.
For ELL students, for example, the control group improved 4.9%, whereas the treatment group improved 15.0%.
Reasons for Concern #1: “Active Controls”
Researchers compared students who used their online curriculum to others who did not. This comparison to a “control group” allows them to say that their curriculum is better than the old way.
However, it’s quite possible that the students in the treatment group responded well simply because they did something different. In other words: perhaps the online curriculum didn’t help students learn, but the change of daily routine did.
The only way to rule out this possibility is to have the control group do something new as well. If the new online curriculum produces more learning than some other new curriculum, then we can reasonably conclude that the curriculum itself made the difference.
After all, in this hypothetical case, we can’t say that just changing something made a difference; both groups changed something.
However, because the control group in this study wasn’t “active” — that is, they didn’t do anything different or special — we can’t be certain why the researchers found the results they did. Maybe it was the interactive online science curriculum. But maybe it was just the change of pace.
The researchers’ own video — linked to above — highlights this concern. One of the teachers who used the online units says that she liked them because teachers simply hadn’t had an organized curriculum in the past. Each teacher had put something together on her own.
If that’s true for the schools in the control group, then this study compares schools that had no consistent, professionally developed curriculum at all with schools that had a curriculum developed by a Oregon University’s School of Education.
In other words: the benefit could come from having any well developed curriculum, not specifically this curriculum. Or not an online curriculum.
We simply can’t know.
Reason for Concern #2: Goodies
There’s no neutral way to say this. Teachers assigned to the online curriculum got a big gift bag. Teachers in the control group got dinky gift bags.
More specifically: teachers in the control group got a $300 bump. (Admittedly, that’s a nice perk for not having to do anything special.)
Teachers who used the online curriculum got $1500. (That’s a REALLY nice perk.) It’s five times as much.
And, they got some cool technology. And, they got extra PD and instructional resources. They even got 2 days of paid subs so they could attend that PD.
For all these reasons, it’s possible that the students did better with the online science curriculum because it helped them learn. It’s also possible that they did better because their teachers were really excited by all the loot they got.
Again, because of the study design, we simply can’t know.
What Should Teachers Do?
In my view, the jury is still out on online curricula.
On the one hand, they seem like a really good idea. On the other hand, the evidence in their favor is, at best, equivocal.
I wouldn’t be surprised if, 20 years from now, we all clapped our hand to our foreheads and asked, what were we thinking?
In other words: we can’t yet look to research to be certain of an answer. If you have an opportunity that appeals to your teacherly instincts, give it a try.
If it works, let me know. If not, don’t worry that you might be doing something wrong. After all: humans evolved learning face to face. It’s just possible that the e-version of that experience will never be as good.
