Study finds chatbot crisis responses depend on how conversations are presented

Liu, J., Chen, X., Cai, J., Luo, X., Xiao, W., & Deng, J. (2026). Context presentation and native conversation structure in large language model responses to escalating suicide risk: A controlled multimodel evaluation. Frontiers in Psychiatry, 17, Article 1962098. https://doi.org/10.3389/fpsyt.2026.1962098

Summarized by Emily Buss

About the Study

A young person may disclose serious distress gradually rather than state an immediate concern in an opening message. For mentoring programs considering AI tools, this raises a question about whether a chatbot can recognize changing risk across a conversation. Liu and colleagues examined whether responses differed according to how earlier disclosures were presented.

The researchers tested ChatGPT, Gemini and Doubao using 12 fictional conversations with 5 user turns each. There were 4 conversations with direct suicide-risk signals, 4 with implicit or inconsistent signals and 4 high-distress conversations without suicidal intent. The main evaluation produced 540 responses. No patients or young people participated, and the chatbot replies were not used to guide anyone’s care.

Each conversation was tested in 3 formats. One presented only the current user message. Another presented the user’s statements together in a fresh conversation with an instruction to consider all available information. The third retained a continuous conversation containing both user statements and chatbot replies. Experts established reference standards before testing. Raters assessed risk recognition, direct safety questions, urgency and timely recommendations for real-world intervention.

Key Findings

Reconstructed prompts performed better on the primary outcome. Timely appropriate intervention occurred in 87.5% of reconstructed cumulative conversations, compared with 41.7% of isolated-message conversations and 50.0% of continuous conversations. Each percentage was based on 24 model-by-risk-conversation combinations, nested within 8 independent risk scenarios. These are benchmark results, not rates of successful intervention with people.

The main contrasts survived adjustment. Reconstructed cumulative presentation exceeded isolated messages by 45.8 percentage points. Continuous conversations performed 37.5 percentage points below reconstructed prompts. Both comparisons remained statistically significant after correction for the primary tests. Continuous conversations did not differ significantly from isolated messages, and differences in major safety errors were not statistically significant.

Results differed across systems. The exploratory comparison between continuous and reconstructed conversations was most unfavorable for Doubao, smaller and uncertain for Gemini, and absent for ChatGPT. The pooled finding should not be used to claim that every chatbot responded in the same way. False risk recognition and excessive escalation in the control conversations were numerically higher with reconstructed prompts, but those differences were inconclusive.

The format comparison did not isolate a cause. Reconstructed prompts also changed message placement, information concentration, fresh-conversation status and instructions to integrate earlier disclosures. The study cannot establish that previous chatbot replies caused poorer performance or that adding conversation history alone improved safety.

A system may retain every previous message yet fail to make evolving risk information sufficiently salient at the moment of decision. (Liu et al., 2026, Discussion, Section 4.2)

Implications for Mentoring

Programs assessing AI tools should examine responses across developing conversations, including how the system handles indirect disclosures and when it recommends human help. A reassuring answer to an isolated test message does not establish safe performance during an ongoing exchange. Structured updates to a system’s assessment of risk are one design approach the authors propose, but this study did not test that approach.

The authors identify several limitations. The evaluation used only 8 independent risk scenarios and 4 controls. User statements followed a fixed sequence and did not adapt to the chatbot’s replies. Testing covered 3 commercial systems through consumer interfaces during a limited period, with limited repeat testing. Results may change as models and interfaces are updated.

The study did not measure whether anyone sought help, received care or experienced a change in mental health. It therefore cannot establish safety for adolescents or college students. For mentoring programs, the practical implication is to retain clear human responsibility for responding to distress and to evaluate conversational performance without treating transcript storage or fluent language as evidence of clinical safety.