Benchmarking Speech Recognition Models for Medical Consultations in Latin American Spanish: A Comparative Evaluation with Fine-Tuning

Fine-tuning did not improve the performance of Whisper Large v3, the top open-source speech-to-text model for Latin American Spanish medical consultations, compared to its baseline.

• Why it matters: Accurate transcription of medical conversations in non-English languages is essential for effective AI medical scribes, yet evaluations in Latin American Spanish are limited, hindering development.
• What they did: Ten speech-to-text models, including open-source and close-source options, were evaluated on Latin American Spanish medical consultation videos, with one video held out for testing, using multiple error and similarity metrics.
• The result: Whisper Large v3 and its fine-tuned version achieved the best performance among open-source models, with the fine-tuned model not surpassing the original; close-source Gemini-2.5-pro performed best overall, supporting tailored AI solutions for Spanish-speaking medical contexts.