Clinically Grounded AI-Scribing in Psychotherapy: Benchmarking LLMs Against Expert Documentation in the iCARE Framework

GPT-4o-mini achieved the highest overall scores among 11 LLMs in generating clinically valid psychotherapy notes using the iCARE framework, with Gemini Pro excelling in interpretative tasks.

• Why it matters: Effective mental health documentation requires capturing nuanced clinical context, which current AI systems often miss, risking inadequate support for clinicians and patient care.
• What they did: Researchers developed the iCARE format and created the iHOPE dataset of 174 therapy sessions, then benchmarked multiple LLMs using automatic metrics and a new human-evaluation framework called TRACE.
• The result: Findings show that while models vary in strengths, all struggle with temporal reasoning and lexical overlap, highlighting the need for structured formats like iCARE to enable trustworthy, therapist-centered AI documentation.