Exploring the Potential of Large Language Models for Automated Safety Plan Scoring in Outpatient Mental Health Settings
Overview
This paper introduces the Safety Planning Intervention Fidelity Rater (SPIFR), an automated tool utilizing large language models to assess the quality of Safety Planning Interventions in outpatient mental health settings. The study evaluates GPT-4, LLaMA 3, and o3-mini on their ability to analyze key components of safety plans using deidentified data from New York. Results indicate that LLaMA 3 and o3-mini outperformed GPT-4, suggesting potential for providing clinicians with timely feedback on intervention fidelity.
- ID
3e9b75dcfbda4eea- Source org
- PubMed Central
- Author
- Not recorded
- Published
- Not recorded
- OA status
- Not recorded
- DOI
- 10.1101/2025.03.26.25324610
- PMID
- Not recorded
- PMCID
- Not recorded
- Citations
- Not recorded
Tags
Verification
- identityok via crossref · score 1 · checked_at unknown
- legitimacykeep via unknown · score 0.95 · checked_at unknown
- linkok via http-check · 2026-07-28T03:58:44.050956+00:00
Cite this entry
BibTeX
@article{allodium:3e9b75dcfbda4eea,
title = {Exploring the Potential of Large Language Models for Automated Safety Plan Scoring in Outpatient Mental Health Settings},
journal = {PubMed Central},
doi = {10.1101/2025.03.26.25324610},
url = {https://doi.org/10.1101/2025.03.26.25324610}
}RIS
TY - JOUR TI - Exploring the Potential of Large Language Models for Automated Safety Plan Scoring in Outpatient Mental Health Settings PB - PubMed Central DO - 10.1101/2025.03.26.25324610 UR - https://doi.org/10.1101/2025.03.26.25324610 ER -
APA
Exploring the Potential of Large Language Models for Automated Safety Plan Scoring in Outpatient Mental Health Settings. (n.d.). PubMed Central. https://doi.org/10.1101/2025.03.26.25324610
Related entries
- Automated Safety Plan Scoring in Outpatient Mental Health Settings Using Large Language Models: Exploratory Study
- Use of a Conversational Agent for Training Mental Health Professionals in Suicide Safety Planning: Pilot Feasibility and Acceptability Study
- Assessment of suicide risk in mental health practice: shifting from prediction to therapeutic assessment, formulation, and risk management
- Computer Administered Safety Planning for Individuals at Risk for Suicide: Development and Usability Testing
- Safety planning-type interventions for suicide prevention: meta-analysis
- The experiences of consumers, clinicians and support persons involved in the safety planning intervention for suicide prevention: a qualitative systematic review and meta-synthesis
- A Prompt Engineering Framework for Large Language Model–Based Mental Health Chatbots: Conceptual Framework
- Predicting Suicide Attempts and Suicide Deaths Following Outpatient Visits Using Electronic Health Records
- Safety Planning on Crisis Lines: Feasibility, Acceptability, and Perceived Helpfulness of a Brief Intervention to Mitigate Future Suicide Risk
- Feasibility and user evaluation of HopeBot: An LLM-powered conversational chatbot for depression screening