A Holiday Present for the Teacher/Skeptic (in Beta)

Teachers who rely on research to inform our teaching–presumably, that’s YOU–routinely hear that we must be skeptical.

“Don’t just believe everything you hear. When someone says that their suggestion is ‘brain based,’ you’ve got to kick the tires.”

Yes. Of course. But: how EXACTLY do we do that? What’s the most effective method for skepticism?

Pens and Laptops

Let’s take a specific example.

You’ve probably heard “research shows” that handwritten notes are more effective than notes taken on laptops. That is: students who take notes by hand remember more than those who take notes on their computers.

If you hunt down the source of that information, you’ll almost certainly end up at Mueller and Oppenheimer’s wittily titled study The Pen is Mightier than the Laptop. It made a big splash when it came out in 2014, and its waves have been lapping over us ever since.

Once you find that source, your thought process might go like this:

Step 1: “Look! Research shows that handwritten notes are superior. I shall forbid laptops forthwith!”

Step 2: “Wait a minute…I’ve been told to be skeptical. Just because Mueller and Oppenheimer say so (and have research to support their claim), I shouldn’t necessarily believe them.”

Step 3: “Hmmm. How exactly can I be skeptical?”

So, here’s my holiday present for you: a website that makes effective skepticism noticeably easier…

Check the Scitation

The website scite.ai leads with this catchy slogan: “making science reliable.”

At least, it’s helping make science reliabler. Or, more reliable.

Here’s how. Surf over to the website, put in the name of the article, and press the magic button.

Scite will then tell you…

…how many later studies have confirmed its findings,

…how many simply mention its findings,

…and how many contradict its findings.

In this case, you’ll discover that 24 studies mention Mueller and Oppenheimer’s study, 1 contradicts it, and 0 confirm it. That’s right. Zero.

So, according to Scite, you’ve got as much research encouraging laptop notes as you do decrying them. But: one of those studies is remarkably famous. And, the other simply isn’t known.

Next Steps

What should you do with this initial information?

At this point, I think the obvious answer is that we don’t have an obvious answer.

Probably, you want to keep looking for further evidence on both sides of the case.

You might find this summary over at the Learning Scientists, where Dr. Megan Sumeracki walks through the nuances and complexities of the research.

You might also find my own article arguing that the Mueller and Oppenheimer research makes sense only if you believe that students can’t learn to do new things. (That’s a strange belief for a teacher to have.) If you believe students can learn new things, then their own data suggest that laptop notes ought to be better.

At a minimum, I hope, you’ll feel empowered in your skepticism. Now you–unlike most people who quote Mueller and Oppenheimer–have a broader picture of the research field. You can start using your judgment and experience to guide your thinking.

An Important Caveat

I don’t know how long scite.ai has been around, but it’s in beta. And, truth be told, it’s not wholly reliable.

For instance: in 2011, Ramirez and Beilock did a study showing that writing about stress before an exam can reduce that stress (for anxious students).

In 2018, Camerer et al tried and failed to replicate those results (and several other studies as well).

When I searched on Ramirez’s study in scite, it showed only one contradiction: a study about “aural acupuncture.” In other words: scite missed an important non-replication, and included an irrelevant finding.

So, you shouldn’t use this website as your only skepticism strategy.

But, as of today, you’ve got one more than you did before. Happy Holidays!

If you’re looking for other skepticism strategies for the holidays, check out this work by Blake Harvard.