Using evidence to do the most good, even when it reveals an “inconvenient truth”
By Patrick T. McCarthy, William T. Grant Foundation
Our supply of evidence about What Works, though still too small, is beginning to grow. I want to focus here on how we use evidence—how we put it to work to help more of our nation’s children and families achieve their full potential. Let’s focus on two important aspects of this question: building the bridge from programmatic evidence to population-level impact and reacting wisely to evidence that reveals an “inconvenient truth.”
Most evidence-building research zeroes in on a particular programmatic intervention and tells us whether it achieved desired results with a specific group at a given place and time. When we are lucky, we see those results replicated in different settings, giving us even firmer ground for saying that this intervention works. Such evidence can help policymakers and program managers make wise decisions about where to invest their scarce dollars so they will do the most good. That is very, very good, but if we stop there we leave unrealized a lot of the potential of good evidence to drive great outcomes for all of our nation’s children and families.
To get to population-level scale, we need to embed the best of What Works into the large public systems that serve not a few hundred children or families, but thousands upon thousands. In part, successfully embedding evidence-based programs in large systems requires them to be adapted to the way the systems work so that referral pathways are smooth, contracting provisions are appropriate and the program has secure access to a dedicated funding stream. A greater and ultimately more important challenge is to draw lessons from the evidence and apply those insights to what public systems do and how they do it. When that happens, those systems can achieve better outcomes for all of the children and families they serve.
To do this, we have to look across multiple evaluations to see patterns, clusters and commonalities. If, for instance, you look at the evidence from the multiple evaluations of Multi-Systemic Therapy, Functional Family Therapy, the Oregon Treatment Foster Care model, and similar programs, common success factors begin to emerge. They all place a lot of emphasis on training and supporting their staff to effectively engage families who often are in crisis and hostile to outsiders at the very point that staff arrive on the scene. There is no moment more critical to the outcome of an intervention than these early steps to create the conditions in which therapeutic work can succeed. And there is no one for whom getting those first relationship steps right is more critical—and more challenging—than public system workers who arrive on a family’s doorstep talking about taking one of their children away or locking up one of their teenagers. If we can isolate the critical success elements common to evidence-based programs and build those into the training and supervision of frontline workers, we should begin to see every family that encounters one of these systems having much better odds for success.
The second issue related to using the evidence we are accumulating is how we act on evidence that challenges prevailing paradigms, especially when such programs don’t just not work, they actually do harm. The most egregious example we have as a country are youth prisons. We have known for decades and without a shadow of a doubt that these facilities, which contradict everything we know about adolescent development, cause harm to young people and create more rather than less crime in communities. Contrary to all available evidence, we continue to lock up two to three times more young people than any other nation. We are beginning to hear calls for change from more and more quarters, and we see political will and leadership emerging. To encourage these efforts and to spark even more, in a recent TEDx talk I committed the Casey Foundation to work with any state that makes the decision to do the reform work and make the investments necessary to successfully and safely close these factories of failure and replace them with programs and facilities that help young people get back on track.
Evidence doesn’t turn itself into policy, especially when it contradicts prevailing paradigms or entrenched funding streams. If we are serious about a What Works movement, we can’t allow ourselves or other decision makers to pick and choose which results we want to act upon. Philanthropy can help by investing in developing and testing alternatives, supporting evidence-driven advocacy, and underwriting technical assistance to evaluators and decision makers.
The conversation about What Works continues, thanks in part to other contributors in this series, including Ron Haskins and Gordon Berlin, as well as many others in the field. I hope this ongoing conversation includes attention to what happens as a result of the evidence. The futures of our nation’s young people depend on it.
This is a repost from William T. Grant Foundation. For original post, please click here.
