Triage Advice From General-Purpose Language Models in Health and Everyday Situations

General-purpose language models recommended professional referral far more often for health than non-health scenarios, with referral at 74.7% vs 46.3% across 53 configurations.

• Why it matters: Referral when contact is unnecessary could increase demand on stretched health services, so it matters whether model advice matches what clinicians judge as needed.
• What they did: In a cross-sectional analysis of 4,346 API- or locally run responses from 53 model configurations to 82 investigator-written prompts (41 health, 41 non-health), referral was coded as unconditional or conditional on future change or doubt and compared with clinician and trained rater judgments.
• The result: When raters said contact was unnecessary, 64.5% of health responses still recommended a professional vs 27.6% of non-health (OR 17.08), with most excess being conditional referral (46.3% vs 21.0%); the health referral gap also grew from 20.5 to 32.2–32.3 percentage points in newer releases.

AI summary of the abstract · may contain errors · not medical advice · read the paper.