Automated Safety Plan Scoring in Outpatient Mental Health Settings Using Large Language Models: Exploratory Study
Overview
This exploratory study evaluates the use of large language models to automatically score the quality of written safety plans in outpatient mental health settings. The research compares three LLMs—GPT-4, LLaMA 3, and o3-mini—against 266 deidentified safety plans to assess their ability to evaluate key components like warning signs and coping strategies. The findings suggest that LLaMA 3 and o3-mini can provide timely feedback on plan fidelity, potentially aiding clinicians in implementing suicide prevention interventions more effectively.
- ID
d217e5cd3dcb4a13- Source org
- PubMed Central
- Author
- Not recorded
- Published
- Not recorded
- OA status
- Not recorded
- DOI
- 10.2196/79010
- PMID
- Not recorded
- PMCID
- Not recorded
- Citations
- Not recorded
Tags
Verification
- identityok via crossref · score 1 · checked_at unknown
- legitimacykeep via unknown · score 0.95 · checked_at unknown
- linkok via http-check · 2026-07-28T03:58:44.050956+00:00
Cite this entry
BibTeX
@article{allodium:d217e5cd3dcb4a13,
title = {Automated Safety Plan Scoring in Outpatient Mental Health Settings Using Large Language Models: Exploratory Study},
journal = {PubMed Central},
doi = {10.2196/79010},
url = {https://doi.org/10.2196/79010}
}RIS
TY - JOUR TI - Automated Safety Plan Scoring in Outpatient Mental Health Settings Using Large Language Models: Exploratory Study PB - PubMed Central DO - 10.2196/79010 UR - https://doi.org/10.2196/79010 ER -
APA
Automated Safety Plan Scoring in Outpatient Mental Health Settings Using Large Language Models: Exploratory Study. (n.d.). PubMed Central. https://doi.org/10.2196/79010
Related entries
- Exploring the Potential of Large Language Models for Automated Safety Plan Scoring in Outpatient Mental Health Settings
- Use of a Conversational Agent for Training Mental Health Professionals in Suicide Safety Planning: Pilot Feasibility and Acceptability Study
- Computer Administered Safety Planning for Individuals at Risk for Suicide: Development and Usability Testing
- Safety planning-type interventions for suicide prevention: meta-analysis
- Assessment of suicide risk in mental health practice: shifting from prediction to therapeutic assessment, formulation, and risk management
- The experiences of consumers, clinicians and support persons involved in the safety planning intervention for suicide prevention: a qualitative systematic review and meta-synthesis
- A Prompt Engineering Framework for Large Language Model–Based Mental Health Chatbots: Conceptual Framework
- Feasibility and user evaluation of HopeBot: An LLM-powered conversational chatbot for depression screening
- Safety Planning on Crisis Lines: Feasibility, Acceptability, and Perceived Helpfulness of a Brief Intervention to Mitigate Future Suicide Risk
- Suicide Safety Planning: Clinician Training, Comfort, and Safety Plan Utilization