Research Alert: New Study Explores Program Factors That Influence Early Match Closures

By Mike Garringer (MENTOR National), Reprinted from the National Mentoring Resource Center

We are happy to announce that some innovative original research developed by the National Mentoring Resource Center has been accepted into the journal Prevention Science and published earlier this year. Research Board members Sam McQuillin (Univ. of South Carolina) and Michael Lyons (Univ. of Virginia) led this research project, which applied cutting edge machine learning techniques to examine predictors of early match closure in the responses of a national mentoring program survey.  Among all of the program characteristics examined, the frequency of ongoing training and match support was the one factor that stood out as being a solid predictor of whether a program’s matches lasted as long as intended.

This research used data from MENTOR’s 2016 National Mentoring Program survey. Most studies of matches within programs follow specific mentor-mentee pairs to see if they last the intended duration if fizzle out early, as well as the personal characteristics and circumstances that may have contributed to their longevity or early closure. While instructive, these studies rarely examine the role that program practices and staff actions play in the course of those matches. This study is one of the few that examined things at the program level—in fact, this examined 29 potential program practices or features that might influence match longevity across over 1,400 unique mentoring programs nationally.

Drs. McQuillin and Lyons used an innovative form of machine learning analysis called Bayesian Additive Regression Trees. While the science behind this technique is complicated, in layperson terms, this method allows a computer to analyze many, many factors that might influence a particular outcome, essentially comparing each factor’s influence, both individually and in almost infinite combinations with others, on a given outcome. In this case, the outcome of the percentage of a program’s matches that lasted their intended duration.

While the results of this analysis contained a lot of “noise” (meaning that it was hard for individual factors to stand out clearly given how many things likely influence match longevity), there were some strong hints as to what best predicted a mentoring program’s matches lasting as long as they were supposed to. The authors note:

“…we found that the reported monthly frequency of ongoing training and support appears to be the best predictor of premature match closure within the available data set, outperforming 29 other predictors.”

The authors also found that programs with larger budgets, more staff per youth, and those serving higher percentages of high academic achievers also tended to have higher percentages of matches going the distance. Those serving higher percentages of youth in the juvenile justice system had less likelihood of avoiding premature closures.

The authors are careful to note that there is a lot of variability in what seems to predict early match closures and that no one program factor is a magic bullet in terms of addressing this issue. But this research also suggests that spending a lot of time checking in with matches and offering robust ongoing training and support to mentors may help reduce problems within matches, deepen mentor confidence and commitment, and increase the utility and enjoyment of the mentoring experience in ways that help participants persist.

The Research Board of the NMRC is thrilled to see analyses funded by this project enter into the scientific literature in this way. If you would like to read more about this analysis, please see this blog post by Dr. McQuillin that talks about this analysis and other research into the importance of mentor training: Improving Mentoring by Improving Mentor Training.

Full-text copies of the Prevention Science article are available through the publisher.

McQuillin, S.D., Lyons, M.D. A National Study of Mentoring Program Characteristics and Premature Match Closure: the Role of Program Training and Ongoing Support. Prev Sci (2021). https://doi.org/10.1007/s11121-020-01200-9        

To access this article, click here.