When Experience Contradicts Research: The Problem with Certainty

A friend recently told me about his classroom experience using mindfulness to promote thoughtful and effective writing.

Young girl at school practicing yoga on a mat

He started the year by explaining the benefits of mindfulness to his students. After that introduction, he began each class period with five minutes of mindful silence.

Although he wasn’t running a research study, he kept two kinds of “data.”

First: his own impression is that students got appreciably better at adding insight, depth, and detail to their writing.

For instance, instead of saying “the mood was optimistic,” they might write “the sun came out” or “I could hear lively music in the background.”

They also “got stuck” less often during the writing process.

Second, he surveyed his students at the end of the year.

He got LOTS of positive responses. One wrote:

I was surprised by how much I looked forward to meditation as the weeks went on, it helped calm me down before big assignments (like practice exams or actual tests) or just gave me a breather during stressful moments.

Another:

I thought meditation was very helpful in class this year because it helped me focus on my clarity of mind. I especially liked it before writing essays because it relaxed me and helped my thoughts flow clearer I think.

A third:

I would start the year by doing it everyday. I’ve started implementing it in my home life and have felt more present and conscious not only my daily interactions but also my thought process and decision making.

I could go on quoting like this for a few pages.

Based on this experience, my friend asked me what research shows about the effects of mindfulness in the classroom…

What Research Shows…

Alas, research into the classroom benefits of mindfulness doesn’t obviously align with my friend’s experience.

Yes, we do have some encouraging research about the ways that mindfulness can reduce stress.

Yes, we do have some correlational research showing a relationship between mindfulness and academic accomplishment.

But my honest opinion is that — so far — we don’t have a strong enough research pool to make inclusion of mindfulness programs a “research-supported” practice in schools.

In particular, we have an ENORMOUS recent study (over 8000 students!) showing that mindfulness training provided students with NO BENEFITS AT ALL, and perhaps (very small) increases in the likelihood of a few problems.

I’m game for the idea that mindfulness training might help students and teachers. But I don’t see enough consistent, high-quality research findings to champion the practice myself.

A Friend’s Quandry

So, a quandry:

My friend’s EXPERIENCE suggests that “brief mindfulness exercises help.”

But RESEARCH suggests that “mindfulness training doesn’t necessarily do anything.”

What should he do?

Equally important: what should YOU do if research suggests that one of your teaching practices a) doesn’t help, or b) might actually hurt?

Let me suggest a few steps.

STEP ZERO: even before we begin answering this question, I think it’s essential to admit that it’s both a challenging and an essential question. (So challenging and important that I wrote a book about it.)

Someone might say to you, “there’s an obviously easy answer to that question.” I think that person is wrong.

STEP ONE: recognize that both kinds of knowledge have their own benefits and weaknesses.

For instance, research helps us see long-term effects that we teachers most often miss.

We know that “short-term performance is an unreliable indicator of long-term learning.” (Thanks, Dr. Nick Soderstrom (on Twitter @NickSoderstrom).)

So, research can help us see when things that feel really good in the classroom right now don’t actually produce the long-term benefits we’re hoping for.

Likewise, research helps us overcome our biases. My friend works REALLY HARD to make his mindfulness intervention work. He devotes LOTS of time to it. (So do his students!)

NO WONDER he sees all the benefits! Well…research can help us see past that motivated reasoning.

Not So Fast…

At the same time, a teacher’s classroom experience provides insights that research just can’t spot.

Teachers combine variables. Researchers isolate variables. That is: we see combinations that researchers rarely explore — combinations like “daily mindfulness + writing.”

Also: research always exists within “boundary conditions.”

A research study done with 3rd grade Montessori students learning long division might — but might not! — apply to dyslexic 11th graders studying history at a military academy.

Unless we have SO MUCH research on a particular topic, a research-only perspective might miss the places that a technique DOES work because we’ve seen that it DOESN’T help in all these other places.

Teachers — however — might discover those places.

Don’t Stop Now

Okay, so we know that this quandry is an important question and requires complex answers (step 0); and we know that research and experience provide separate and useful kinds of knowledge (step 1).

What’s next?

STEP TWO: Get specific.

In that 8000+ person study: what exactly did they do? And: how well does “what they did” align with “what my friend did”?

In the 8000+ person study, they had students practice mindfulness for 10 weeks. They wanted to know if a) the students would keep doing mindfulness on their own after the 10 weeks, and b) if their mindfulness practice would help them — or not — according to 28 different criteria.

The answers were a) “nope, not really” and b) “nope, not at all.”

But: OF COURSE the students didn’t get the benefits of mindfulness (that’s b) because they didn’t continue the mindfulness exercises at home (that’s a).

Notice, however, that this research doesn’t align with my friend’s strategy. His students DID continue the mindfulness because he started every class with time for mindfulness.

True: students who don’t practice mindfulness don’t benefit from it; but my friend’s students might benefit because they had time to practice it.

In other words: that big study shouldn’t necessarily discourage my friend, because his strategy differs from their strategy in meaningful ways.

STEP THREE: Combine humility with determination.

Here’s the trickiest part.

As I’ve just argued, this big study might not apply to my friend’s approach.

AND: my friend’s “biased” perspective (we ALL have “biased” perspectives) might make it difficult for him to recognize the shortcomings in his approach.

For this reason, I think we have to push ourselves relentlessly to balance humility (“I should really focus on and respect research guidance!”) with determination (“My classroom experience is valuable and I should give it weight in my decision making!”).

But, gosh, that’s a difficult balancing act.

It’s tempting to pick one side or the other:

I shall do what research tells me!

or

My training and instincts matter most!

Instead, we should strive to give both sources of knowledge their due…and always doubt our own certainties.

An Excellent Example

Note, by the way, that my friend was doing just that.

After all, his own classroom experience — and his students’ enthusiastic feedback! — gave him all sorts of reasons to be entirely confident.

He might easily have said “research, schmeesearch” and gone ahead with his mindfulness routine.

Instead, he looked for a reason to doubt his own certainty; that is, he asked me what research has to say … knowing that it might not support his experience. (He had, after all, just finished reading my book on evaluating research-based teaching claims.)

He now has to decide the best way to procede. And: in my view, he will do so all the more effectively because he allowed himself to doubt.

In this field, certainty is the enemy of improvement and excellence.

Leave a Reply

Your email address will not be published. Required fields are marked *