The Chronicle of Evidence-Based Mentoring
  • X
  • Instagram
  • LinkedIn
  • Editors Blog
  • Natural Mentoring
    • Coaches
    • Teachers
    • Other Non-Parent Adults
  • Mentoring Practice
    • Culture and Context
    • Ethical Considerations
    • Matching
    • Mentor Recruitment and Retention
    • Program Evaluation
    • Termination & Closure
    • Training
  • Program Models
    • Group Mentoring
    • K-12
    • Paraprofessionals
    • Post-Secondary
    • Workforce Development
    • Youth-Initiated Mentoring
  • Improving Outcomes
    • Increasing Engagement
    • Supportive Accountability
    • Targeted Mentoring
    • Technology Delivered Interventions
  • Special Populations
    • Children of Incarcerated Parents
    • Immigrants and Refugees
    • Juvenile Offenders
    • Military
    • Students
    • Youth in Foster Care
    • Youth with Mental Health Needs
  • Profiles
  • Webinars
  • Search
  • Menu Menu

Reflections on a major new evaluation of community-based mentoring

November 30, 2022/in Editors Blog/by Jean Rhodes
By Jean Rhodes

Mentoring experts Carla Herrera, David DuBois, Janet Heubach, and Jean Grossman have just published a major new randomized controlled trial of the effects Big Brothers Big Sisters of America (BBBSA) Community-Based Mentoring (CBM) Program on the social-emotional, behavioral, and academic outcomes of youth. The study included over 700 9- to 14-year-olds who were enrolled in two West coast BBBSA agencies and followed over a 13-month period. The study serves as an important next step following the landmark evaluation of BBBSA CBM, which was conducted in the mid-1990s and helped to launch today’s mentoring movement in the U.S. (Tierney et al., 1995). As in the earlier study, each participating youth was assigned to be immediately eligible for a Big or put on a waitlist (in this case 13 months). Similar to the original evaluation, the study assessed a wide range of youth-reported outcomes. In addition, it included several parent-reported outcomes and created composite indices which reflected the average of youth- and parent-reported outcome measures.

Results: The evaluation found “significant differences between treatment and control groups on only one of the eight youth-reported outcome measures: depressive symptoms [though a small effect (d = .146)]. For the remaining youth-reported outcome measures, estimated effects were in directions favoring the treatment group for all but two measures: misconduct and skipping school.” For parent-reported outcomes, analyses revealed statistically significant differences favoring the treatment group on four of the six tested measures: emotional symptoms, peer problems, conduct problems, and the SDQ Total Difficulties Score. The study also found significant differences favoring the treatment group on the parent-report and combined youth- and parent-report composite indices. Overall, effect sizes were small, ranging from .138 to .253.

As the authors note that “the current study found few significant impacts in the outcome domains it has in common with the earlier P/PV study of BBBSA CBM (Tierney et al., 1995).” They note several factors that might account for these differences.

  1. Shorter matches: By the 13-month follow-up, “only about half of the treatment youth had received at least 12 months of mentoring, and the average match length was less than 10 months, whereas the follow-up period for the original P/PV study (Tierney et al., 1995) was 18 months, at which point, matched youth had met with their mentors an average of 12 months (Grossman & Rhodes, 2002).
  2. Smaller sample: The P/PV trial was based on a larger sample and thus was in a better position to detect small effects on program outcomes.
  3. Lower dosage: While youth in the treatment group reported fairly strong relationships with their mentors, only 15 percent met with their mentor 3 or more times in most of the months they were matched. In contrast, over 70 percent of youth in the original P/PV study met with their mentor at least three times a month, and 45% met at least once a week (Tierney et al., 1995).
  4. Implementation issues:
    1. Training: Program practices varied across the matches involved in this study. This was especially true for initial “pre-match” mentor training (beyond basic program orientation), which was received by only about one in three mentors.
    2. Monthly support: Implementation of monthly support contacts was also inconsistent, with support contacts not taking place in at least one-third of the months that matches were active for mentors, youth, or parents.

A few reflections:

This was an extremely impressive study. RCT’s are the gold standard, the instruments were well validated, and the discussion section offers a master class in nuanced interpretation. See article. 

One interesting idea that they raise is that small average benefits “may reflect the non-targeted nature of the BBBSA CBM program, which focuses broadly on relationship development, rather than on any particular outcome. A recent meta-analysis by Christensen et al. (2020), in fact, found that targeted mentoring programs yielded substantially larger effect sizes on the outcomes they targeted than did non-targeted programs (typically tested on a broader set of outcomes). Perhaps one of the strengths of non-targeted programs like BBBSA CBM is their ability to yield “small” benefits, but across a wide range of outcomes, whereas targeted programs may yield larger benefits, but only in the specific areas targeted by the programs.”

I would argue, however, that small effects across a broad range of positive outcomes can be scaled and achieved more economically through after-school and other PYD programs (4-H, Boys & Girls Clubs). For example, a recent meta-analysis of the effects of after-school programs among youth with marginalized identities (Christensen et al., 2022), found larger youth-reported effects overall than CBM (g = 0.205  vs. d = .115, respectively). Since only about 5% of youth will ever be assigned a formal mentor, and many youth face long waitlists, an argument could be made for expanding access to/concentration of adults in PYD and after school programs and deploying formal one-on-one mentors to help youth more in need of targeted care and support. After school programs receive more federal funding and most mentored youth are already involved in PYD programs (Jarjoura et al., 2018). Thus, if the goal is to create more youth-adult bonds and engage in PYD activities that modestly effect a range of outcomes, the most efficient structure would involve practice and policy changes that lead to more caring adults in ASPs to improve youth-staff ratios, staff burnout and turnover, and ideally, support longer tenure of staff through better pay.

Another interesting finding that the authors note was that there were many more significant parent-reported (vs. child-reported) impacts. The authors note that this pattern of findings also emerged in a Friends of the Children evaluation (which found only parent effects) and highlight the value of collecting additional sources of data when assessing program effects. Whatever the reason, it is interesting and encouraging that, across two rigorous studies, parents are seeing value in their children’s enrollment in mentoring programs.

 

 

 
Tags: After School, BBBS, Goal Setting, Training
Share this entry
  • Share on Facebook
  • Share on X
  • Share on X
  • Share on WhatsApp
  • Share on Pinterest
  • Share on LinkedIn
  • Share on Tumblr
  • Share on Vk
  • Share on Reddit
  • Share by Mail
https://www.evidencebasedmentoring.org/wp-content/uploads/2022/11/Screen-Shot-2022-11-30-at-9.46.48-PM.png 661 1024 Jean Rhodes https://www.evidencebasedmentoring.org/wp-content/uploads/2023/03/chron-logo.jpg Jean Rhodes2022-11-30 21:53:432024-08-12 16:42:07Reflections on a major new evaluation of community-based mentoring
You might also like
New study highlights the five mentoring competencies (5MCs) in mentor training
Building students’ resiliency networks: A conversation with Brian Lightfoot
Research Assistant Position: Help with research, evaluation, and training and technical assistance projects in the area of youth mentoring.
More than fun and games? New study explores how mentors strike a balance between fun and goal-oriented activities
Researcher urges more talk with youth about emotions
Phone call or text?: New study explores youth’s “Supportive Accountability” preferences
All you need is “deep level” similarity: Predictors of quality mentoring matches
Understanding the link between early adverse experiences and diminished school engagement

Editor’s Blog

  • AYPF Presentation: Referrals to Mentoring for Youth Involved in the Juvenile Justice System - Chronicle of Evidence-Based Mentoring
    How Single-Session Interventions Could Transform MentoringMay 14, 2025 -
  • “I get it and I care:” The power of validationApril 28, 2025 -
  • Capturing the “Beauty” of Group MentoringApril 13, 2025 -
  • Navigating the OCEAN: Leveraging the Big Five Personality Traits for Effective Mentor–Mentee MatchingApril 2, 2025 -
  • The need to belong: New meta-analysis highlights the importance of a “sense of belonging”March 23, 2025 -

Tags

After School Arts & Culture Asia BBBS Class Closure Coaching College Culture E-Mentoring Elementary School Employment Equity Ethnicity Europe Gender Goal Setting Higher Education High School Immigration Incarceration K-12 Medicine Mental Health Mentor Benefits MentorPRO MHapps Mobile Apps Motivational Interviewing Politics Race Reciprocal Recruitment Religion Retention Social Justice STEM Supportive Accountability TDI Termination Training University Violence Vocational Volunteers
Jean Rhodes, Ph.D.
Jean Rhodes, Ph.D.
Editor in Chief
Megyn Jasman
Megyn Jasman
Assistant Editor
Ramya Ramadurai
Ramya Ramadurai
Associate Editor

Topics of Interest

After School (114) Arts & Culture (106) Asia (50) BBBS (58) Class (90) Closure (94) Coaching (84) College (129) Culture (90) E-Mentoring (22) Elementary School (59) Employment (12) Equity (90) Ethnicity (90) Europe (57) Gender (55) Goal Setting (194) Higher Education (49) High School (59) Immigration (91) Incarceration (112) K-12 (59) Medicine (56) Mental Health (61) Mentor Benefits (54) MentorPRO (59) MHapps (113) Mobile Apps (113) Motivational Interviewing (170) Politics (54) Race (90) Reciprocal (54) Recruitment (95) Religion (46) Retention (95) Social Justice (90) STEM (2) Supportive Accountability (107) TDI (113) Termination (94) Training (138) University (129) Violence (55) Vocational (12) Volunteers (144)

Sponsors

  • MentorPROFebruary 21, 2023 -

    MentorPRO is a leading digital tool platform to support educators to build developmental relationships with their students in easy and fun ways.

  • Along
    AlongMarch 30, 2022 -

    In support of our continued conversation on how to support today’s youth, this issue of the Chronicle of Evidence-Based Mentoring is proudly sponsored by Along, a free digital tool designed to support educators to build developmental relationships with their students in easy and fun ways.

  • National Mentoring Partnership
    MENTOR: The National Mentoring PartnershipNovember 12, 2014 -

    MENTOR: The National Mentoring Partnership (MENTOR) is the unifying champion for expanding quality youth mentoring relationships in the United States. For nearly 25 years, MENTOR has served the mentoring field by providing a public voice, developing and delivering resources to mentoring programs nationwide and promoting quality for mentoring through standards, cutting-edge research and state of the art tools.

  • Academic Web PagesAugust 6, 2012 -

    Academic Web Pages is the leading provider of customized websites for researchers, centers, nonprofits, and universities. AWP designed and has contributed generously to the creation of the Chronicle of Evidence-Based Mentoring.

Older and Wiser
The Chronicle of Evidence-Based Mentoring. All rights reserved. Site By Academic Web Pages.
  • Editors Blog
  • Natural Mentoring
  • Mentoring Practice
  • Program Models
  • Improving Outcomes
  • Special Populations
  • Profiles
  • Webinars
Scroll to top
We use cookies to analyze site traffic and provide a better browsing experience. If you continue to use this site, you consent to our use of cookies. Learn more about our updated Privacy Statement that supports the new EU data protection laws. Ok