Even older and wiser: Guessing again about how to improve mentoring

 

Good tests kill flawed theories; we remain alive to guess again.

Karl Popper

By Jean Rhodes

“Older and wiser: New ideas for mentoring in the 21st Century ” was released in paperback last week. As I reflect on this milestone, I must acknowledge that, in the 2.5 years since the book was published, some ideas have aged better than others.

Overall, the book argued for a more targeted, evidence-based approach to mentoring. Shortly after the book was published, a meta-analysis by Christensen et al. (2020), found that, indeed, targeted mentoring programs yielded substantially larger effect sizes on the outcomes they targeted than did non-targeted programs. Although another new study shows that skills-based approaches must be balanced with youth-centered relationship building, the data continues to support the promise of more targeted approaches to mentoring. But how do we get there? In the book, I argued that targeted mentoring can be achieved in several ways, including specialized, embedded, and blended approaches to mentoring. As the numerous studies continue to show, smaller specialized mentoring programs are well-poised to target specific subgroups (e.g., youth aging out of foster care and unaccompanied refugees), special risks (e.g., depression, anxiety, and peer rejection), and/or specific goals (e.g., STEM training and applying to college) and achieve powerful outcomes. But, when larger programs try to specialize, they can only hit the mark with a subset of mentees.

Lessons Learned 

Although I focus on the lessons we’ve learned through work at the Center for Evidence-Based Mentoring, excellent studies by colleagues in the fields of mentoring, youth development, and education have also been published in the ensuing years, many which have been highlighted in the Chronicle.

1. A narrow focus on mental health/educational apps was misguided. To test the idea that mentors could provide supportive accountability to youth using evidence-based mental health and educational apps like HeadSpace, we developed supportive accountability tech platforms–a K-12 version, MentorHub, and a customization of MentorHub, MentorPRO, for college and workforce development programs. These platforms were originally designed with the narrow (and ultimately misguided) focus of helping mentors to track and encourage their mentees’ use of third-party apps. Given the opportunity, mentors and mentees do not want to work together on mental health and other apps, instead preferring recreation and, if goal-focused, working together on broader goals and activities–both in person and online. Thus, although both our supportive accountability tech platforms have been well-received as a means of easily connecting mentees, mentors, and supervisors, setting and tracking goals and challenges, and providing resources and training, there hasn’t been much uptake of third-party apps. Consequently, we have expanded the definition of supportive accountability to include skills and goals more broadly. In fact, we successfully piloted a higher education model and found significant associations between engaging in peer mentoring through our college-customized version of the platform and improved academic and wellness outcomes (Werntz et al. 2023).
2. In other contexts, supervised practice and supportive accountability can improve intervention effects. There aren’t yet enough studies to test these concepts in mentoring, so we explored supervised practice and supportive accountability in the context of skills training and mental health interventions.
   • Christensen et al., (in press) synthesized results from five meta-analyses and found a modest, but significant, overall effect of supervised practice compared to unsupervised practice (SMD = 0.22), with internalizing behavior yielding the largest effect. As we noted, our findings suggest that supervised practice “has the potential to significantly improve the effectiveness of a range of skills-based interventions.”
   • To test the idea of blended mentoring (in which mentors provide supportive accountability to digital mental health interventions (DMHIs)), we conducted a meta-review (Werntz et al., 2023) in which the results of 31 meta-analyses (505 unique primary studies) were analyzed. Almost half (n=22, 48%) of the studies showed that human-supported DMHIs were significantly more effective than unsupported DMHIs while only 9% (4/45) of effect sizes showed that unsupported DMHIs were significantly more effective. As we note, “Our findings highlight the potential of human support in improving the effects of DMHIs. Specifically, evidence emerged for stronger effects of human support for individuals with greater symptom severity.” So, we haven’t given up on mentors supporting DMHI’s. The idea is still solid, and we suspect that we’ll have better traction with older adolescents and young adults.
3. Embedded mentoring is a logistical challenge. In the book, I noted that embedded programs “hold promise but require a high level of community integration and logistical finesse.” Turns out this was an understatement. Certainly some programs have it down to a science, like Boston Partners in Education. But starting from scratch has been difficult. To test embedded mentoring, we teamed up with a DC youth mentoring program, and a team of experts at Georgetown (Dr. Matthew Biel), Harvard (Dr. John Weisz), and UT Austin (Dr. Sarah Kate Berman), to test the idea that mentors could reinforce what was being learned in school-based mental health clinics. This experiment is still underway but has hit many logistical hurdles–from human subjects approval delays to difficulties coordinating different program schedules and getting parent and program buy-in. Nevertheless, we remain optimistic about this model.
4. Many programs still lack the incentives and tools to “shift tasks” to mentors. Since the book’s publication, my colleagues and I have continued to advocate a range of roles that mentors can play in bridging current gaps in mental health, outlining strategies for a “task shifting approach.” Of course, moving the needle on specific mental health outcomes is not easy. A new study showed that only about 39% of youth who are in psychotherapy for depression improve. Ultimately, however, the largest proportion of youth in the US are served by programs that continue to embrace nonspecific “friendship” models, providing their volunteers with less than optimal hours of pre-match/ongoing training and continuing to produce relatively small effects. A recent meta-analysis of the effects of after-school programs among youth with marginalized identities (Christensen et al., in press), found larger youth-reported effects (g = 0.21) than those recently reported in nonspecific, community-based mentoring. Thus, if the goal is to create more youth-adult bonds and engage in positive-youth development activities that modestly affect a range of outcomes, the most efficient structure would be to place more caring adults in after-school settings to improve youth-staff ratios, staff burnout and turnover, and ideally, support longer tenure of staff through better pay.