Do results apply to my program?: External validity, generalizability, and transferability refer to fairly similar ideas.

Screen Shot 2014-03-10 at 10.13.05 AMby Adar Ben-Eliyahu, Ph.D.

Technically speaking there are slight differences in the definitions presented below:

  • External validity– to what extent do the study findings apply to other programs not studied within the particular study?
  • Generalizability  – is this true for other similar programs?
  • Transferability – can this be applied to other programs that are different?

All of these answer questions such as:

  • Does research on one Big Brother Big Sister program (e.g., Boston) apply to other Big Brother Big Sister programs (e.g., New Mexico)?OR
  • Can findings from a study on foster-care youth program be used to inform about other mentoring programs such as Campus Corps?

How to be a critical thinker in applying research to your program?

As you read through studies, ask yourself the following questions:

  • How is my program similar to the one that is studied?
  • How is it different?
  • Who are the mentors and mentees in each program?
  • How old were each?
  • How recent is this study?
  • Do the study recommendations apply to my program?

Example:

A recent study in Campus Corp found that grouping mentors and youth into mentor families has benefits for both mentors and mentees.

How is my program similar to the one that is studied?

–       My program uses university students as well.

How is it different?

–       Mine is a 9-month program

Who are the mentors and mentees in each program?

–       The mentors in both programs are students, though in my program students are slightly older than Campus Corp.

–       The mentees have similar characteristics though identified through different processes.

How recent is this study?

–       This study is recent and is up to date with current recommendations

Do the study recommendations apply to my program?

–       I could group mentors into groups of 3-4, although that would mean that I would need to schedule their meetings at the same time and place.

–       I could do this once a month or every 3-5 weeks.

Remember: Not every detail or recommendation can be transferred from one program to another.  It is important to understand how studies can be applied and findings adapted for your program’s needs.