SraVaani 适配尼泊尔语-英语混说语音识别，WER 从 48.6% 降至 17.1%
Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model
做低资源语音识别的可以看看这篇。SraVaani 全量微调后 WER 干到 16.3%，比微调版 Whisper large-v3-turbo 还低一点。它处理数字符号的 tokenizer 妥协方案和消融数据，对做尼泊尔语、混说语音这类项目很有参考价值。
一篇 arXiv 论文研究了如何将 449M 参数的混合 TDT-CTC 语音基础模型 SraVaani 适配到尼泊尔语与英语混说的场景。团队构建了覆盖六个尼泊尔语音频域、共 172.8 小时的语料库，针对预训练 tokenizer 不支持数字和常见符号的问题，生成数字的尼泊尔语与英文读法并用 zero-shot ASR 输出选择合理写法，而不是替换 tokenizer 或丢弃录音。
实验显示，全量微调把测试集 WER 从 48.6% 降到 17.1%，12 个 epoch 后达到 16.3%，略优于微调后的 Whisper large-v3-turbo 的 16.4%。消融实验表明冻结 encoder 会让 WER 升至 25.5%，剔除含数字的音频则使数字测试语音的 WER 从 14.3% 升到 21.3%。
团队还在 FLEURS 基准上用原始标注和混合文字写法两套参考做了评估，发现转录规范可以解释部分域外性能损失，但不是全部。这为低资源、混说语言的 ASR 适配提供了数据处理和微调策略参考。
Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model
Nepali speakers regularly mix English words into conversation, but multilingual speech recognisers often struggle with language switches and the writing script used for English words. We study how to adapt SraVaani, a 449M-parameter hybrid TDT-CTC model, using a 172.8-hour corpus covering six Nepali speech domains. Its pretrained tokenizer does not support digits and several common symbols. Rather than replace it or discard these recordings, we generate Nepali and English readings for numbers and use zero-shot ASR output to select a plausible form. Full fine-tuning reduces test word error rate (WER) from 48.6% to 17.1%; a 12-epoch run reaches 16.3%, compared with 16.4% for fine-tuned Whisper large-v3-turbo. Freezing the encoder increases WER to 25.5%, while excluding numeric clips raises WER on numeric test speech from 14.3% to 21.3%. We also evaluate on FLEURS using both its original references and mixed-script alternatives. The comparison shows that transcription conventions explain part, but not all, of the out-of-domain performance loss.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include SraVaani, Whisper, 语音识别, 微调.
Source attribution: curated from arXiv cs.AI (@Jeevan Neupane, Janardan Bhetwal, Kshitiz Paudel, Anup Aryal, Bipin Bashyal).