Profiles in Mentoring: Mackenzie Hart on Just-in-Time Mentor Training

Mackenzie J. Hart, Ph.D., is an Assistant Professor in the School Psychology Program at the University of Texas at San Antonio. Dr. Hart’s research focuses on empowering youth in low-resource environments by building motivation for positive change and expanding access to evidence-based interventions. She is particularly interested in training and equipping paraprofessional providers, such as youth mentors, with motivational interviewing skills to strengthen non-traditional service delivery models. We recently had the opportunity to speak with Dr. Hart about her recent paper on innovative training solutions for enhancing mentor effectiveness in school-based settings!

Chronicle (C): What inspired you to explore just-in-time training as a method for building mentoring skills, and how did your own experiences with mentoring influence this study?

Mackenzie Hart (MH): Interest in exploring just-in-time training as a method for building mentoring skills developed fairly organically, building off previous trials/ versions of AMPED. AMPED, the program referenced in the manuscript, is a goal-directed and relationship-based mentoring program that pairs mentors (primarily undergraduate volunteers) with middle school students. AMPED modules were adapted/ developed from existing evidence-based treatments, such as Homework, Organization, and Planning Support (Langberg et al., 2012) and Motivational Interviewing (Miller & Rollnick, 2023). However, early evaluations of AMPED showed no benefit of the program by measures of student connectedness and behavior, and some academic measures actually decreased (i.e., students in a control group outperformed students who were randomly assigned to AMPED). Our research team determined that mentors had been (a) doing things we asked them not to do, such as finding faults with mentees’ behaviors, grades, etc., and (b) not doing things we asked them to do, such as partnering with their mentees in a collaborative manner. These findings suggested to our research team that mentors’ initial training was insufficient; mentors require additional and ongoing support to effectively deliver evidence-based interventions. Just-in-time training has a robust and growing evidence base in other areas, such as medicine and healthcare, which made it a promising approach to explore in our context.

My own experience with mentoring added to my excitement about just-in-time training, as I can recall some first-hand experiences prior to graduate school where I believe I could have benefitted from just-in-time training. In fact, as an undergraduate student, I served as a mentor for middle and high school students in Dorchester! This experience helped me understand that working with children and adolescents inherently imbues responsibility to (a) help them, if possible, and (b) not harm them, to the best of the mentor’s ability. However, most mentors—through no fault of their own—do not enter programs equipped to respond to mentees’ various needs. The just-in-time training format could allow mentors to gain these needed, select skills “on demand,” rather than leaving them feeling helpless and overwhelmed, as I remember feeling in some instances.

C: Your study found that mentors improved most in reflective listening skills. Were there aspects of the video design that you think made that skill especially teachable?

MH: My hypothesis as to how and why mentors improved most in reflective listening skills centers on the repetition each just-in-time training video afforded in that area. While each video focused on a specific skill or activity in the AMPED curriculum, the importance of asking open-ended questions, making affirmations, and offering reflections and summaries was frequently reiterated, with each just-in-time training video offering an example/ providing a model of how reflective listening skills could present in each module. I am very excited about a forthcoming paper that offers a content analysis of the just-in-time training videos referenced in this study, which supports the reasoning herein.

Further, through each of the just-in-time-training videos, our team sought to balance didactics with demonstrations. In other words, we provided overviews and instructions regarding mentee-mentor interactions, emphasizing to mentors who watched our videos that empathic listening and responding are valuable to relationship-building; however, we also incorporated brief role-play examples of what these interactions do and do not “look like.” I think this structure bolstered mentors’ understanding of what to do and what not to do as far as reflective listening, through juxtaposing desired and undesired behaviors.

C:  Given the positive feedback on usability, how do you envision JITTs being scaled or adapted in real-world mentoring programs, particularly those outside of academic settings?

MH: I and others would love to see just-in-time training videos implemented in programs outside of academic partnerships! A large appeal of the just-in-time training format itself is its emphasis on efficiency, scalability, and reproducibility.

Aspects of our video design further sought to position us toward this goal, as our university-based research team partnered with local community members through the development process, asking collaborators to review the plans and scripts for our just-in-time training videos. We believe these conversations helped increase the relevance of video content to mentees and mentors, such as through constructing the mock conversations (e.g., mentee-mentor problem-solving) featured in the videos. We also sought consultation from a larger mentoring organization to maximize likelihood that our just-in-time training videos could be adapted for other programs.

Additional collaboration and work will be needed to actualize these goals of accessibility and dissemination, however. It would be helpful to use an existing, larger infrastructure to distribute just-in-time training materials to other mentoring programs or sites. Moreover, it will be important to support the use of these materials after they are available, as evidenced by findings regarding system climate and system support on the URP-IR measure. In intervention usability research, promoting a positive system climate refers to ensuring alignment between the intervention (i.e., just-in-time training videos) and the existing structures. In our original study, the existing structures were the middle school and the university-based mentoring program, but to scale just-in-time training videos, we will need to attend to other structures (e.g., mental health clinics, mentoring programs, and schools). System support refers to additional help or resources mentors report needing to use the just-in-time training videos, and to implement their corresponding skills confidently and effectively. Some mentors indicated professional development and coaching would be beneficial, which highlights areas for future clinical and implementation research to explore.

Dr. Hart’s paper can be found here