这篇论文告诉你如何微调Whisper搞定低资源语言阿萨姆语的语音识别，效果比零样本好一大截，WER直接降了78%
该论文提出基于Whisper模型的阿萨姆语ASR系统，在Mozilla Common Voice 24.0-Assamese语料库上微调，采用混合精度训练和梯度累积优化资源受限环境。微调后词错误率（WER）降至43.17%，字符错误率（CER）13.18%，与零样本基线相比分别相对提升78.26%和93.10%。语义评估BLEU得分为30.81，METEOR为0.5262，幻觉率降低96.70%，实时因子（RTF）改善32.38%。
Robust Assamese Speech Recognition through Controlled Fine-Tuning of Whisper Models
Developing Automatic Speech Recognition (ASR) for morphologically rich, low-resource languages such as Assamese is challenging due to insufficient annotated speech data. The pretrained Whisper model performs poorly on Assamese speech recognition tasks. This paper presents a controlled, fine-tuned Whisper-based Assamese ASR system trained on the Mozilla Common Voice 24.0-Assamese corpus. A hardware-aware optimized training pipeline is implemented for resource-constrained environments, employing mixed-precision training and gradient accumulation on Tesla 4 Graphics Processing Units (T4 GPUs). The proposed fine-tuned model significantly outperformed the Zero-shot baseline, yielding Word Error Rate (WER), Character Error Rate (CER), Match Error Rate (MER), and Word Infomation Loss (WIL) of 43.17\%, 13.18\%, 43\%, and 64.81\%, respectively, achieving significant relative improvements of 78.26\%, 93.10\%, 57.0\%, and 35.19\% over the baseline. Semantic evaluation of the fine-tuned model also demonstrates notable improvement over a zero baseline, attaining Bilingual Evaluation Understudy (BLEU) and Metric for Evaluation of Translation with Explicit ORdering (METEOR) scores of 30.81 and 0.5262, respectively. Additionally, the predicted hallucination rate and Real-Time Factor (RTF) are substantially improved by 96.70\% and 32.38\%, compared to the zero-shot baseline.