BBBS Ireland releases evaluation results
Summary of key findings
[In this evaluation], (from report) standardised survey measures were used to assess outcomes at 4 time points, or waves,
over a 2-year period for intervention and control groups. The outcomes assessed related to 4 dimensions
– emotional and mental well-being, education, risk and problem behaviour, and relationships and social
support. Youth in both intervention and control groups showed improvements on all measures over the
study period, with the exception of those related to risk behaviour.
The RCT results show that the mentoring intervention was effective in relation to emotional well-being and social support, with statistically significant findings for the intervention versus control group analysis and a minimum effect size of 0.13 at Wave 4 (the end of the study, October 2009). Results for the Children’s Hope Scale were strongest out of the 14 youth outcome measures assessed, with an effect size of 0.22 at Wave 4. The effect sizes for total perceived social support and support from other adults at Wave 4 were both 0.13. There were positive, but non-significant trends in relation to social acceptance, school liking, plans for school and college completion, and drug and alcohol use. Measures of scholastic efficacy and misconduct were also non-significant.
The results of the parent survey measures showed a significant decline in control group parents’ perceptions of their children’s pro-social behaviour over Waves 2 to 4, whereas the change over time for members of the intervention group was positive. The parent strand of the RCT did not show significant evidence of impact in relation to parents’ perception of their children’s academic performance or of an improvement on the strengths and difficulties measure.
In terms of aggregate effect size, control and intervention groups were statistically equivalent in the young people’s survey at baseline, with an average Cohen’s d of 0.01 at Wave 1. According to the criteria developed by Cohen (1988), a small effect is represented by a Cohen’s d of 0.20. An average Cohen’s d of 0.15 was recorded for Wave 2, which rose to 0.19 at Wave 3 and decreased to 0.09 at Wave 4. The effects
recorded are consistent with other studies of mentoring. For example, the study by Tierney et al (1995) of community-based mentoring recorded an average effect size of 0.06 at 18 months post-baseline, at which equivalent time point (Wave 3) an effect size of 0.19 was recorded for the present study. The meta-analysis by DuBois et al (2002) of 55 mentoring studies found an average effect size of 0.14. These
results therefore suggest that the BBBS Ireland intervention had an impact in a similar range to that found in previous studies.
