Using Vision and Large Language Models for Psychological Intervention Competency Assessment: A Pilot Study of Agreement with Clinician Ratings

A Vision Language Model plus Large Language Model pipeline rated ENACT therapeutic competency in 25 simulated therapy videos, with AI-clinician AC1=0.73 and median ICC=0.57.

• Why it matters: Supervision for psychological intervention training is constrained by the time required to evaluate simulated sessions, motivating faster competency assessment from communication signals.
• What they did: Gemini-3.1-Pro annotated 1,558 behaviors in videos and fed them into transcripts, then ChatGPT-6.1-Sol scored against ENACT while clinicians verified VLM-identified behaviors.
• The result: Seventy percent of VLM-identified behaviors were confirmed by both clinicians and 93% by at least one; AI-clinician agreement approached between-clinician AC1=0.75 and may support supplementary evaluation in training.

AI summary of the abstract · may contain errors · not medical advice · read the paper.