Program evaluation is critical to determine whether children’s needs are adequately addressed

Axford, A., Bjornstad, G., Matthews, J., Whybra, L., Berry, V., Ukoumunne, O. C., Hobbs, T., Wrigley, Z., Brook, L., … & Warner, G. (2020). The effectiveness of a community-based mentoring program for children aged 5–11 years: Results from a randomized controlled trial. Prevention Science. https://doi.org/10.1007/s11121-020-01132-4

Summarized by Monica Arkin

Notes of Interest:

• This randomized controlled trial assessed the effectiveness and implementation of a year-long mentoring program for children with behavioral issues in London.
• Analyses found that children in the intervention group did not significantly differ from the control group on parent and teacher ratings of behavior.
• The authors conclude that the mentoring intervention did not affect children’s behavior as intended, and requires changes in design and/or implementation in order to increase effectiveness.

Introduction (Reprinted from the Abstract)

The study, a two-arm, randomized controlled, parallel group, superiority trial, aimed to evaluate the implementation and effectiveness of a 12-month one-to-one volunteer mentoring program designed to improve behavioral and emotional outcomes in children aged 5 to 11 years who have teacher- and parent/carer-reported behavioral difficulties. Participants were 246 children (123 intervention, 123 control; mean age 8.4 years; 87% boys) in five sites in London, UK, scoring in the “abnormal” range on the teacher-rated Strengths and Difficulties Questionnaire (SDQ) Total Difficulties measure and in the “borderline” or abnormal range on the parent-rated SDQ Total Difficulties measure. Randomization on a 1:1 ratio took place using a computer-generated sequence and stratifying by site. Data collectors and statisticians were blind to participant allocation status. Outcome measures focused on parent- and teacher-rated child behavior and emotions, and child-rated self-perception and hope. Intention-to-treat analysis on all 246 randomized participants (using imputed data where necessary) showed that at post-intervention (16 months after randomization), there were no statistically significant effects on the primary outcome—parent-rated SDQ Total Difficulties (adjusted standardized mean difference = − 0.12; 95% CI: −0.38 to 0.13; p = 0.33)—or any secondary outcomes. Results from complier average causal effect (CACE) analysis using the primary outcome indicated the intervention was not effective for children who received the recommended duration of mentoring. Exploratory analyses found no sub-group effects on the primary outcome. The article concludes that the mentoring program had no effect on children’s behavior or emotional well-being, and that program content needs revising to satisfactorily address key risk and protective factors.

Implications (Reprinted from the Discussion)

There was no statistically significant effect on any outcome. Given the high level of need of children at baseline, it is possible that many participants were recruited at a point of crisis, and that this level of need in both arms naturally reduced slightly over time. Effect sizes at endpoint are small and none are statistically significant. Moreover, children in the control arm were eligible to receive services as usual, and it is reasonable to suppose that some of the regular services they received—in particular CAMHS (Child and Adolescent Mental Health Services)—may have contributed to improvements in their outcomes over time. Neither is there evidence of an effect on compliers under the CACE (complier average causal effect) analysis. But given the relatively serious needs of the children at recruitment, the lack of effect may be related in part to what mentors actually deliver and whether program content focuses sufficiently and efficaciously on relevant issues. Chance UK is now engaged in a process of intervention adaptation, testing, and refinement, in large part informed by the results and conclusions of this trial.

To access this article, click here.