Human-calibrated automated intelligibility evaluation of speech neuroprostheses

Fine-tuning Whisper-large-v3 with LoRA on human-transcribed brain-to-voice speech for one participant increased correlation between model WER and human WER from 0.62 to 0.76.

• Why it matters: Intelligibility assessment for brain-to-voice neuroprostheses is hard because synthesized speech lacks ground-truth acoustics, and human transcription is costly and difficult to scale or compare.
• What they did: The team collected ~14.5 hours of synthesized speech with human transcriptions (yielding ~72 hours total) and fine-tuned Whisper-large-v3 using LoRA to make model WER approximate human WER.
• The result: On held-out sentences, fine-tuned WER closely tracked human WER, including speech from unseen decoders, and similar agreement was reached with only 1.5 hours of synthesized speech.