KeyASRBench 发布：用关键词错误率衡量 ASR 专名保留能力
KeyASRBench: Measuring Whole-Phrase Keyword Preservation in Real-World ASR
做语音转写的朋友可以看看这个基准：各家模型 WER 看着挺低，但专有名词的关键词错误率能到 33%，难怪生产环境都要加热词纠错。评测脚本和挑战集都开源了，可以直接拿来测自己的 ASR 方案。
研究团队发布 KeyASRBench，一个中英双语 ASR 评测基准，采用基于出现次数的关键词错误率（KER）来衡量转写文本是否保留人名、产品名、技术术语、版本号等整词短语。基准包含 10 小时模型条件挑战集，共 1,501 条中英文音频片段、4,723 个关键词出现位置，全部采用冻结的整词标注。
评测覆盖 5 个本地推理模型检查点和 2 个官方 API 系统：这些系统的语言级 MER/WER 在 5.31% 到 12.91% 之间，但 KER 高达 13.61% 到 33.81%，说明低序列错误率掩盖了大量关键词丢失。一个不相交的 30 分钟模型盲测对照组 KER 为 10.65%-27.31%，证明关键词错误并非挑战集选择造成的偏差。
这一结果解释了为什么生产级 ASR 系统仍普遍依赖热词列表、上下文偏置、检索增强和事后纠错。对语音产品开发者来说，KER 提供了比 WER 更贴近实际业务的评估维度。
KeyASRBench: Measuring Whole-Phrase Keyword Preservation in Real-World ASR
Public automatic speech recognition (ASR) benchmarks increasingly report low sequence error rates, yet production systems still use hotword lists, contextual biasing, retrieval, and post-correction. This motivates a complementary question: does a transcript preserve selected names, products, technical terms, versions, and relational phrases? We present KeyASRBench, a bilingual pilot that operationalizes occurrence-based Keyword Error Rate (KER) over frozen whole-phrase annotations. The 10-hour model-conditioned challenge set contains 1,501 Chinese and English clips with 4,723 keyword occurrences. Across five locally inferred checkpoints and two official API systems, language-specific MER/WER ranges from 5.31% to 12.91%, while KER ranges from 13.61% to 33.81%. A disjoint 30-minute model-blind control yields KER of 10.65%-27.31%. The control confirms that selected-phrase errors extend beyond challenge-set selection.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include KeyASRBench, ASR, 语音识别, 关键词错误率.
Source attribution: curated from arXiv cs.AI (@Guanzheng Wei, Ziyi Zhao).