When Does Technology Distract Students? The Benefits of Research that...

I spoke with several hundred students last week about research-based study strategies.

As always, students were fascinating to hear about psychology and neuroscience research: for instance, the benefits of retrieval practice.

And, as always, they did not love my alarming news about multi-tasking. My advice goes like this:

“If you want to study less and learn more, do one thing at a time.”

No insta-snap-gram-tweet-flix-chat-tok while you’re studying. One thing at a time.

Since that talk, I’ve found some interesting research about the LIMITS of that advice, so I thought I’d share it here.

Tech Problems, Tech Benefits

Our reasons to worry about technology use during class seem perfectly obvious. If I am paying attention to my tweets, I am not paying attention to my academic work.

Divided attention = less learning. Obviously.

At the same time, we can easily see ways that technology benefits learning.

If — during a lecture — students text one another to reinforce their understanding of the material (“What did the prof just say?”), they might solidify their learning.

If they look up complementary information on the interwebs, their quest might boost their comprehension. (I’ve been surprised how often my students want to do this in class, and I occasionally allow them to do so.)

So, we need a more precise question than “is technology good or bad?” We need to know — under what precise circumstances does it help? Or hurt?

Technology and Higher Order Thinking

For instance: does off-topic texting during a lecture interfere with both “lower order” and “higher order” thinking, as defined by Bloom?

And, by the way, what role does note-taking play?

A study from 2018 explores this question.

The details, of course, get complicated, but the short version goes like this. Students watched a 24 minute lecture about psychiatric diagnoses: PTSD, ADHD, OCD, and so forth. They also took notes.

Some students received and answered off-topic texts during the lecture — one about every 2 minutes.

After the lecture, students took a test.

Some of those test questions focused simply on recalling details: “How long must an individual display symptoms of PTSD in order to be diagnosed?”

The researchers designed these questions to measure knowledge and comprehension — that is, “Bloom’s level 1 & 2.”

Four questions, however, focused on deeper understanding: “Compare PTSD and ADHD. How do these disorders differ? Are there ways in which they are similar?”

That is: these questioned aimed to measure application and analysis: Bloom’s level 3 & 4.

So: what effect did the OFF-TOPIC TEXTS have on the students’ NOTES, and on their UNDERSTANDING?

The Envelope Please

The researchers’ results surprised them — and certainly surprised me.

Unsurprisingly, students distracted by texts took less complete notes.

And, also unsurprisingly, they did substantially less well on the factual questions. Texters averaged a 62 on those questions, while non-texters averaged a 71.  (If you speak stats, the Cohen’s d was 0.64. That’s an impressively large difference, at least to me.)

Here’s the surprise: researchers did NOT find a statistically significant difference between students’ scores on the application and analysis questions.

How do we explain this finding?

First: let’s admit the possibility that texting distractions do interfere with factual recall but not analysis.

Honestly, I would not have anticipated that finding, but it could be true.

Second: perhaps the timing matters. That is: these students took the test immediately after the lecture. Perhaps application and analysis — unlike mere factual recall — require more time for steeping.

That is, if the “higher order thinking skills” had been tested the next day, perhaps we would have seen a difference in those scores.

Third: perhaps the number of questions mattered. Because the researchers asked only 4 application/analysis questions, they might not have had enough data to discern a difference.

Perhaps a greater number of questions would have revealed a difference.

The Big Picture(s)

Based on this research, will I tell students “it’s okay to text during lectures”?

NO I WILL NOT.

Here’s why:

First, facts matter. If off-topic texting interferes with factual learning, that finding itself means that texting during lectures is bad.

Second, taking notes properly (almost certainly) matters. If texting  interferes with good note-taking, that finding itself should dissuade students from doing so.

Third, I’m willing to believe that texting doesn’t interfere with application/analysis, but only if other studies — with more questions and later tests — consistently demonstrate that result.

Another point also jumps out at me from this research. This study contradicts my firmly held belief that multitasking vexes learning.

I genuinely believe that IT’S A GOOD THING when research contradicts my firmly held beliefs.

If research never contradicted my beliefs, then I would never learn anything from it.

In fact, I would never need to look at research because it shows me only what I already know.

Research might prove most useful to us when it contradicts our beliefs.

Who knows, maybe I’ll go back to those students and update my advice…

Waite, B. M., Lindberg, R., Ernst, B., Bowman, L. L., & Levine, L. E. (2018). Off-task multitasking, note-taking and lower-and higher-order classroom learning. Computers & Education, 120, 98-111.