Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios.

Large language models recommend overuse in 5.4% of scenarios based on Choosing Wisely guidelines, primarily for thyroid ultrasound and electrocardiography.

• Why it matters: Overuse of medical tests and treatments can lead to unnecessary costs and patient harm, highlighting the need to evaluate AI tools guiding clinical decisions.
• What they did: Six commercial LLMs generated 4050 responses to 225 scenarios, with 15 recommendations each, assessing their tendency to endorse restraint or action in line with guidelines.
• The result: Overuse was concentrated in two recommendations, with false restraint being rare at 0.6%, suggesting LLMs generally avoid withholding appropriate actions but may over-recommend in specific areas.