New research identifies program practices as predictors of match longevity

Kupersmidt, J. B., Stump, K. N., Stelter, R. L., & Rhodes, J. E. (2017). Mentoring program practices as predictors of match longevity. Journal of Community Psychology, 1, 1-16. DOI: 10.1002/jcop.21883

Summarized by Justin Preston

Introduction

Mentoring relationships have long been associated with positive academic, social, emotional, and behavioral outcomes for youth. These positive findings, though, are tempered by the repeated findings that mentoring, on average, produces only fairly modest outcomes for participating youth. Although the average effect size (or impact) of mentoring programs is modest, there is a great range of variability in the impact a program can have. This means that some programs are having a greater impact than other programs. One aspect of this impact may be captured in the program practices the mentoring organization is using to facilitate the match process.

The present study sought to identify the impact of programs that implemented best-practice guidelines into their programming versus those that did not using the metric of match longevity. This included the extent to which programs utilized the Elements of Effective Practice for Mentoring (EEPM) in their program practices. The main goal was “to explore the prevalence of implementation of program practice benchmarks and standards, as well as the consequences of implementation fidelity on match length,” as the authors state. The authors expected that greater compliance with evidence- and safety-based program practices would be linked to greater success in a program’s ability to foster longer matches between their mentors and mentees.

Methods

Participants in this study included 45 Big Brothers Big Sisters (BBBS) local agencies located across 28 states in the United States. Each of these programs utilized the BBBS Agency Information Management (AIM) software to support their management of the program. These programs ranged dramatically in size, with some working with more than 13,000 matches, while others had less than 30 matches represented in the dataset. All matches were one-to-one, half of the agencies were community-based, and half were site- or school-based. The present study utilized only the community-based programs, leaving a final sample size of 29,708 matches.

Program staff who completed the survey were largely women (91%), White (82%), and had a mean age of 42 (range: 24 to 65). The majority of mentors were women (63%), White (66%), and non-Hispanic (91%). The average age of the mentor was 31.74 years. The majority of mentees were girls (59%), 28% were White, 17% were Hispanic, and, on average, 10.78 years old at the beginning of their match.

The researchers collected data around the demographic characteristics of the sample and programs, the match relationship length, and premature match closure. Premature match closure was considered to occur any time a match lasted less than the 12-month commitment BBBS typically requests. Mentoring matches were considered to be long-term if they persisted 24 months or longer.

Lastly, program practices were assessed using a shortened version of the Elements Quality Improvement Process (EQUIP) measure. EQUIP is a self-assessment measure developed to ascertain the extent to which programs were utilizing the benchmarks and guidelines outlined in the EEPM.

Results

Prevalence of implementation of benchmark practices – Of those programs assessed, implementation ranged between 13 to 21 of 22 practices assessed using EQUIP. Programs utilized the greatest number of standards in their screening, matching, and closure protocols. About one third of all programs implemented the training protocols. Only one program was in full compliance with the guidelines in the EEPM, but this was largely due to BBBS-wide protocols on the frequency of monitoring and support differing from EEPM benchmarks.

Individual program practice standards as predictors of match length – The training standard was the only standard that was independently significantly associated with match length.

Sum total implementation of benchmark and standard practices as predictors of match length – Both the benchmarks and standards were significantly associated with match length, and total number of standards and benchmarks were both significantly different from the baseline projection model.

Grouping programs by level of adherence to the standards – Dividing programs between high adherence (implementing 19 or more of the recommended guidelines, 20% of programs) and low levels of implementation (18 or fewer guidelines, 80% of programs) resulted in a significant difference between the groups. Those that were coded as high implementation performed significantly better in fostering long matches than those in the low implementation group.

Implications for Mentoring Programs

As the authors state, “Overall, the results revealed that the sum total number of benchmarks and standards implemented by mentoring programs was significantly associated with match length, particularly for matches surviving for 2 years or longer. The only individual standard that independently predicted match length was training.” In other words, greater adherence to the guidelines outlined in the EEPM is more likely to put your program in a position to foster long-term mentoring relationships.

This research also highlights the limits of the “buffet-style” method of selecting pieces of evidence-based practices to implement in your program. That is, just incorporating one or two evidence-based practices may be helpful, but you are more likely to see significantly better benefits if the program adopts and adheres to more evidence-based, structured approaches.

It is important to keep in mind, however, that this study was conducted with BBBS programs. While on the one hand, this helped to clarify the impact of adopting the EEPM guidelines and benchmarks by looking at programs who had many of the same common goals, structure, and model, more research needs to be done on the impacts of high-level adoption of evidence-based practices for different types of youth mentoring programs.

Still, this study presents compelling evidence for the importance of moving toward a more rigorous, evidence-based approach on the programmatic side to foster greater success in creating long-term mentoring matches.

To read the original research, click here.