What is Your Evidence-Based Mentoring IQ? The Winner is…
by David DuBois
I apologize for the delay in following up on my “evidence-based mentoring IQ” posting from May. I appreciate the time (and courage!) of all who took the time to try their hand at the quiz. The winner of the free copy of the new Handbook of Youth Mentoring (2nd edition) is F. Parks. If you can kindly contact me with your full name and address, I’ll be pleased to send you a signed copy of the book.
Overall, 45 persons completed the survey. The average score was 6.5 correct (out of 15 possible) with a standard deviation (SD) of 2.4.* Respondents did somewhat better identifying “true” statements (average of 4.4 out of the 8 possible) than “false” statements (average of 2.0 out of 7 possible). The 30 respondents who identified themselves as practitioners had an average score of 6.4; as would be expected, the two researchers who completed the quiz scored higher on the quiz (average of 10.5). Interestingly, the 7 respondents who identified themselves as being in intermediary roles (i.e., “work for local, state, regional or national org providing support to mentoring programs”) scored relatively low on the survey with an average of 6.0. Although the sample size is small and potentially quite non-representative, this finding does suggest to me the value of looking further at the level of familiarity that individuals in such roles have of evidence-based research information. Results could support investment in efforts to strengthen the awareness that those working in intermediary roles – which are vital to successfully connecting research and practice in the field – have of recent research on youth mentoring that may be important to their efforts in supporting the work of mentoring programs.
Below I provide a question by question review of the correct answers for the quiz and associated source information. Questions and comments, as always, are welcome. I hope to repeat this type of quiz on an annual basis, inspired by the positive feedback I have received about it being a fun and useful exercise. Hopefully, whatever your feelings about the value of this particular effort, we can all agree that continuing to look for and try out ways to strengthen the connections between research and practice should be among the field’s highest priorities.
1. A recent* large (several hundred participant) randomized controlled trial (RCT) of the impact of participation in the Big Brothers Big Sisters of America (BBBSA) community-based mentoring program largely replicated the results of the earlier, landmark RCT evaluation of the program in the 1990s by Public/Private Ventures with respect to demonstrating favorable (modest, but statistically significant) effects on youth emotional, social, and behavioral outcomes.
FALSE—see Herrera et al. (2013) full report available at http://www.mdrc.org/publication/role-risk In the random assignment portion of this evaluation, significant program effects were evident for only two outcomes (depressive symptoms and an aggregate measure of positive change), neither of which were part of the earlier P/PV evaluation. Effects were not found on significant outcomes from that evaluation, such as problem behavior, school performance, or parent trust.
2. A recent study of the BBBSA school-based mentoring program found that program effects were similar regardless of whether youth met with mentors at lunch, after-school, or as part of pull-out during the non-lunch part of the school day.
FALSE—see Schwartz et al. (2012) full article available at http://www.rhodeslab.org/files/MeetingTime.pdf Program effects “Among academically at‐risk youth, the impact of school-based mentoring on academic outcomes was moderated by the time during which matches met. Specifically, academically vulnerable youth derived significant academic benefits from mentoring in programs that met after school or during lunch. In programs that met during school as a pullout program, there was no evidence of benefits and some evidence of negative effects on academic outcomes.”
3. A recent study reported promising evidence that a school-based mentoring program for bullied youth reduced the extent to which participating youth were victimized (bullied) by their peers.
TRUE—see Elledge et al. (2010) full article available at http://massmentors.org/sites/default/files/School-based%20mentoring%20as%20selective%20prevention%20for%20bullied%20children-%20a%20preliminary%20test.pdf “Children in the Lunch Buddy program (n = 12) were paired with a college student mentor who visited twice each week during the spring semester of an academic year. Also participating were 24 matched-control children; 12 were from the same school as Lunch Buddy children (“Same” controls) and 12 were from a school different from that of Lunch Buddy children (“Different” controls). Results indicated that compared to Different control children, Lunch Buddy children experienced significantly greater reductions in peer reports of peer victimization from fall to spring semesters. Lunch Buddy children and mentors viewed the relationship as positive, and parents and teachers were very satisfied with Lunch Buddy mentoring.”
4. A recent long-term follow-up of an RCT of a youth mentoring program found that program participants, as an entire group, had significantly lower overall levels of involvement in crime during adulthood relative to non-mentored controls.
FALSE—see O’Donnell & Williams (2013) – see abstract and full site at http://cpx.sagepub.com/content/1/1/54.short “Among those with an arrest before program referral, significantly fewer [program] participants [relative to control group members] were arrested as adults; however, among those without an arrest before the mentoring program, significantly more female participants were arrested as adults.” Due to these contrasting effects, for the average participant, there was no evidence of a significant program effect in either direction.
5. Recent evidence suggests negative effects of rematching youth in school-based mentoring programs.
TRUE—see Grossman et al. (2012) full article available at http://www.rhodeslab.org/files/ToTime2.pdf ”At the end of the year, youth in intact relationships showed significant academic improvement, while youth in matches that terminated prematurely showed no impact. Those who were re-matched after terminations showed negative impacts.”
6. Recent research found that evidence of positive effects of volunteer-based one-to-one community-based mentoring on youth outcomes (in a variety of areas) tended to be greatest among youth who exhibited relatively high levels of indicators of individual risk (emotional, behavioral, and/or academic difficulties) when entering programs.
TRUE–see Herrera et al. (2013) full report available at http://www.mdrc.org/publication/role-risk “The study’s findings as a whole thus suggest that the benefits of volunteer-centered community-based mentoring are not confined to youth with particular types or levels of risk. There were some exceptions to this general pattern—most notably a trend toward somewhat stronger and more consistent benefits for youth who were relatively high on individual but not environmental risk.”
7. Recent findings are largely consistent in indicating fewer benefits when mentoring relationships last less than one year for youth participating in programs where mentors are asked to make a commitment of 1 year or more.
FALSE– see Herrera et al. (2013) full report available at http://www.mdrc.org/publication/role-risk “In general, the expected association between total time mentored and stronger estimates of program benefits was not evident.” Although total time mentored is not the same as length of the youth’s first mentoring relationship, findings were very similar when focusing on the latter.
8. Initial findings from the ongoing RCT of the Friends of the Children program (a program that utilizes paid mentors for high-risk youth) do not indicate significant benefits for participants two years after entry into the program.
TRUE—see presentation by Eddy et al. at the 2013 meetings of the Society for Research on Child Development (link to this presentation is not presently available)
9. Recent evidence indicates that mentoring relationships in school-based mentoring programs are less likely to be intact at the end of the school year when programs have matches meet together at the same time and place.
TRUE—see Grossman et al. (2012) full article available at http://www.rhodeslab.org/files/ToTime2.pdf “Having all the matches meet in the same place appears to have a near-significant negative effect on maintaining the relationship (OR = .64, p = .07)”
10. Supporting mentors in advocacy roles with youth has been found to be predictive of greater program effectiveness.
TRUE—see DuBois et al. (2011) full article available at http://www.psychologicalscience.org/index.php/publications/journals/pspi/mentoring.html Greater positive program effects were found in this meta-analysis when programs supported mentors in assuming advocacy roles with youth. Interestingly, a more recent meta-analysis by different investigators arrived at the same conclusion.
11. Recent evidence has indicated superior academic outcomes for ethnic minority youth who are matched with same-race/ethnicity mentors in community-based mentoring programs compared to those matched with White mentors.
FALSE—there is, however, evidence that having mentors support ethnic minority youth in developing beliefs in the potential for their academic abilities to grow and develop (i.e., that they are not “fixed”) is beneficial—see Good, C., Aronson, J., & Inzlicht, M. (2003). Improving adolescents’ standardized test performance: An intervention to reduce the effects of stereotype threat. Journal of Applied Developmental Psychology, 24, 645-662.
12. Recent findings provide support for a model in which the competence of staff (as rated by their supervisors) contributes to the mentoring relationships they support having greater quality and longevity.
TRUE—see Herrera et al. (2013) full report available at http://www.mdrc.org/publication/role-risk “We then examined whether case manager competence (as reported by supervisors) was linked with mentors’ experiences of support, including the extent to which the case manager seemed truly concerned about how well the match was going, had shared important information about the youth and had offered suggestions for activities. We found that, indeed, supervisors’ reports of case manager competence were associated with mentors’ reports of the quality of the support they received. Mentor reports of greater support quality were, in turn, associated with youth reports of better relationship quality (feelings of closeness, youth centeredness and growth/goal orientation) and with the length of the match at follow-up. These findings suggest that the associations between case manager competence and match outcomes may result, at least in part, from how the mentor experiences the support provided by this staff member.”
13. Preliminary evidence suggests that asking youth to nominate or identify their mentors from their existing social networks is problematic because of both an inability of a substantial proportion of youth to identify suitable mentors and the tendency for a significant percentage of those adults asked to decline the opportunity to enter into a more formalized mentoring relationship.
FALSE—see Schwartz et al. (2013) full article available at http://www.rhodeslab.org/files/YIM_AJCP.pdf and summary of Spencer et al. (in press) on the Chronicle website (https://www.evidencebasedmentoring.org/an-in-depth-look-at-youth-initiated-mentoring/) “Most youth were able to identify a mentor with ease and if they received assistance it was commonly from a parent.”
14. A recent meta-analysis found that cross-age peer mentoring programs, overall, when rigorously evaluated have been less effective for promoting mentee outcomes in comparison to mentoring programs that make use of adults as mentors.
FALSE—see DuBois et al. (2011) full article available at http://www.psychologicalscience.org/index.php/publications/journals/pspi/mentoring.html “Similarly, although programs typically have utilized adult volunteers and focused on cultivating one-to-one relationships, those that have engaged older peers as mentors or used group formats show comparable levels of effectiveness.”
15. Some recent research suggests that there are youth whose genetic make-up predisposes them to benefit more from mentoring than other youth.
TRUE—see Shanahan et al. (2007) full article available at https://sociology.byu.edu/Erickson/SiteAssets/Shanahan%202007.pdf “The over-arching pattern suggested by these data for the white and black males is clear: the link
between DRD2 A1+ and continuation beyond high school is fully attenuated by mentors who are teachers, but males with A1+ are less likely to have a mentor who is a teacher then males with A1–. As with all studies documenting gene–environment interplay, replication is essential… Positron emission tomography shows that the presence of the DRD2 TaqI A1 allele is associated with lower density dopamine receptors and a reduced number of dopamine binding sites in the striatal region in the brains of healthy, living humans (Jonsson et al., 1999; Oohjalainen et al., 1998)….These regions of the brain have been associated with the reward system, impulse control, planning, and related behaviors, and the dopaminergic system has been linked to reward-seeking behavior in humans.” So, it seems mentors, at least those who are teachers, can be of particular help to youth whose genetic profiles may be associated with vulnerability to difficulties with impulse control and related behaviors.
*If you completed the survey and would like to receive a record of your answers, please just email me (dldubois@uic.edu) and I can send them to you.
