Student chatbots are super friendly and helpful: That’s what worries me

By Jean Rhodes

As universities grapple with surging demand for student services, artificial intelligence has emerged as a seductive solution. Chatbots, armed with sophisticated natural language processing, are increasingly deployed to handle everything from basic inquiries to complex tasks like course selection and degree planning. These digital assistants, available 24/7, promise to alleviate the burden on human advisors and accommodate diverse student schedules.

Yet, as we rush to embrace this technological panacea, we risk overlooking its potential pitfalls. While AI can efficiently handle routine queries, it may inadvertently exacerbate existing inequalities. Underserved students, who often face greater barriers in navigating the “hidden curriculum” of college, are more likely to be directed to these digital interfaces. This raises concerns about the quality of support these students receive and their opportunities to build crucial social capital. Perhaps most disconcerting is the growing sophistication of these AI systems. As historian Yuval Noah Harari warns, chatbots may soon “mass-produce intimacy,” simulating close relationships with uncanny accuracy. This artificial rapport could satisfy students’ need for belonging, potentially diminishing their motivation to seek support from real people.

The implications of this trend are profound. College is not just about acquiring knowledge; it’s a rich context for developing social skills and networks that are vital for future success. Our research has consistently shown that students who learn to identify and recruit faculty, staff, older peers, and other forms of social capital earn higher GPAs and forge stronger relationships with faculty and staff. A rigorous study led by Sarah Schwartz provides compelling evidence for the importance of human connections. Students who took a course in which they learned about the value of social capital and how to recruit support in college, showed increased academic self-efficacy and help-seeking attitudes and behaviors one year after training. Moreover, they had more social capital and mentoring support following the intervention. This study’s findings were further reinforced by an analysis of six years of administrative data from UMass Boston, the third most diverse university in the US. Students who took the social capital course had nearly double the first- and second-year retention rates as their peers and were 3.23 times more likely to graduate in four years. Remarkably, this course emerged as the single strongest predictor of student retention and graduation among all factors evaluated (Hersch et al., under review).

AI has an important role to play in scaling effective student support–e.g., including analyzing data and providing personalized suggestions. But rather than student-facing, chatbots could be positioned as co-pilots to peer mentors, advisors, faculty, and other frontline supports. This will increase the chances that students will cultivate a network of real relationships in college that can serve as bridges to new opportunities, connections, and success.