Early Education Program Evaluation: “Differential Susceptibility” to Success

AdobeStock_99365599_Credit

Show me the Money

As most parents, teachers, and education policy folks know well, early childhood education is expensive. Whether federally-funded, state-funded, or family-funded, preschool and structured early care generally operate on a pretty tight budget. They also generally operate on pretty high hopes: academic achievement, personal growth, reduced delinquency, and much more.

And they should! As Ralph Waldo Emerson wrote, “there is no knowledge that is not power.” We certainly need to maintain high expectations for youth to get the most out of their academic careers. As well, we should expect the programs that we invest in to set children up for the success that they promise.

Show us the Results

So what happens when we don’t see those hopes result in program outcome data; in particular, at the state- and federally-funded program level?

  • Do we launch an investigation into what went wrong?
  • Do we take the money away?
  • Do we blame the teachers, or parents, or school districts?

The “what now?” of underwhelming achievement is a challenging road to venture down. For some context, check out my colleague Austin’s recent blog post regarding a newly published study looking at the infamous fadeout effects in Head Start preschools.

Unfortunately, questions of whom to blame have dominated much of the “what now?” conversation over the years. Yet some studies, like the one Austin discussed, are trending in a new, positive direction for developmental and educational research alike.

Let’s Re-think ‘Results’

This new genre of studies does two things. First, it looks at such factors as fidelity to a particular program’s plan. Let’s take Head Start as an example. Researchers will ask: how well and how often are Head Start’s specialized strategies actually being implemented in classrooms?

Second, and most important, these studies don’t stop there. Instead, they go on to broaden the idea of an outcome to include measures of mental health and social growth, and the image of a learning environment to include the home and child care centers.

Broadening what we think achievement is, and where we think learning happens, is an important movement. Of course, many developmental psychologists have been advocating for this broadening for years. Social psychologist Urie Bronfenbrenner, for example, began studying ways in which intra- and inter-person factors affect learning back in the 1970’s. But the merging of research questions that focus on individual context with research questions that focus on school program evaluation is an exciting new empirical endeavor.

Differential Susceptibility

An endeavor that we stand to gain a lot from. One way that these new context+program evaluation research questions are making an impact is in studies of early achievement and differential susceptibility (DS).

DS is a theoretical model that aims to understand why some things affect some people differently. In developmental research, DS refers to children who are more behaviorally or biologically reactive to stimuli and, as a result, more affected by both positive and negative environments. [1]

Study 1

Let’s look at a longitudinal study conducted by researchers at Birkbeck University of London. [2] They investigated the effects of early rearing contexts on children of different temperaments. The following data was collected from 1,364 families:

predictive measures

  • parents reported the temperament of their child at 6 months (general mood, how often they engage in play behavior, how well they transition to a babysitter, etc.);
  • parenting quality (i.e. maternal sensitivity) was assessed at 6 and 54 months during laboratory and home observations;
  • quality of child care (e.g. daycare) was assessed at 6, 15, 24, 36, and 54 months via observation

outcome measures

academic achievement, behavior problems, teacher-child conflict, academic work habits, and socio-emotional functioning were assessed regularly between 54 months and 6th grade

Results showed that children who had a difficult temperament in infancy were more likely than children who didn’t to benefit from good parenting and high-quality childcare. They also suffered more from negative parenting and low-quality child care.

Most pronounced was the finding of differential effects for child care quality. Here, high quality care fostered fewer behavior problems, less teacher-child conflict, and better reading skills while low quality care fostered the opposite — but, only for those children who had a difficult temperament.

The takeaway: children that had a difficult temperament in infancy were differentially susceptible to quality of parenting and child care. For them, the good was extra good, and the bad was extra bad.

Study 2

Researchers at Stanford University engaged high- and low-income kindergartners in activities designed to elicit physiological reactivity (measured by the amount of the stress hormone cortisol in their saliva). [3] In other words, the children completed activities that were difficult and kind of frustrating. They also completed a battery of executive function assessments.

It turns out that children who displayed higher reactivity (more cortisol) during the activities were more susceptible to their family’s income. That is, family income was significantly associated with children’s EF skills — but only for those children with high cortisol response. Highly reactive children had higher EF skills if their family had a higher income, but lower EF skills if their family was lower income.

The takeaway: children that were highly reactive when faced with challenging activities were differentially susceptible to their family’s resources. Their EF was particularly strong if their family had high income, yet particularly weak if their family had a lower income.

Evaluating Program Evaluation

How is being mindful of phenomena like differential susceptibility helpful when we receive the news that children made no special long-term gains after being enrolled in a publicly-funded program?

First, we should recognize that we may have set ourselves up for some disappointment at the outset if we assumed that all children would be equally susceptible to the positive effects of home or school interventions.

Of course, at school entry, we don’t necessarily know which students are arriving with difficult temperaments. Or whether their child care environment has exacerbated or buffered it. Which means that we’re also not going to be able (practically or ethically) to separate students by level of disadvantage in order to decide which program they should be enrolled in. So let’s just accept that we’re going to see some variation in individual outcomes.

Let’s also remind ourselves that variation is not necessarily reflective of an ineffective program. At Head Start, for example, it is probably safe to speculate that most families are juggling some amount of stress, financial instability, and social tension. And according to the DS model, students who are predisposed to be highly reactive will be hit hardest by these things. As a result, reactivity is probably going to interfere with their reaching what we define as success. But DS also tells us that they have the most to gain from a nurturing, consistent environment.

So let’s not take the money away. Let’s hold off on passing the blame around. And let’s not refer to these data as something going “wrong”. Let’s instead look at the students who continue to struggle and ask what contextual factors  — such as a child’s weak self-regulation skills and their parent’s inability to address it in the way their teacher wants because they work two jobs — are at play.

I’m no gambler, but if we can figure those things out, and commit to doing something about them, then I say we double-down when it comes to funding.

 

References

  1. Ellis, B. J., Boyce, W. T., Belsky, J., Bakermans-Kranenburg, M. J., & van IJzendoorn, M. H. (2011). Differential susceptibility to the environment: An evolutionary–neurodevelopmental theory. Development and Psychopathology, 23, 7–28. doi:10.1017/S0954579410000611 [link]
  1. Pluess, M., & Belsky, J. (2010). Differential susceptibility to parenting and quality child care.  Developmental Psychology, 46, 379-390. [link]
  1. Obradovic, J., Portilla, X. A., & Ballard, P. J. (2015). Biological sensitivity to family income: Differential effects on early executive functioning.  Child Development, 87(2), 374-384. doi: 10.1111/cdev.12475 [link]
tags: / category: L&B Blog

Leave a Reply

Your email address will not be published. Required fields are marked *