这个研究用DiffusionGemma做语音识别，并行解码8步就达到6.6%错误率，只训练了0.16%的参数，挺高效的。
本文提出用离散扩散语言模型DiffusionGemma（26B MoE）替代自回归解码器进行语音识别。方法冻结Whisper编码器提取声学特征，通过轻量投影器和低秩适配器将音频映射到模型空间，仅训练约42M参数（占0.16%）。研究发现自然训练目标梯度无法有效传递，引入连接主义时序分类（CTC）损失后成功训练。在LibriSpeech test-clean上达到6.6%词错误率，并行解码仅需8步且与语速无关，单适配器训练六种语言（评估了英语、印地语和普通话）。
Audio-Native Speech Recognition with a Frozen Discrete-Diffusion Language Model
Automatic speech recognition is dominated by autoregressive decoders that emit one token at a time. We ask whether a discrete diffusion language model can transcribe speech instead, refining a whole transcript in parallel over a small number of denoising steps. We train an audio-native interface for DiffusionGemma, a 26B mixture-of-experts model that generates text by uniform, random-token discrete diffusion rather than the absorbing-mask scheme common to recent diffusion language models. A frozen Whisper encoder supplies acoustic features, a lightweight projector maps them into the model embedding space, and low-rank adapters let the frozen backbone attend to the new modality. About 42M parameters are trained, which is 0.16 percent of the backbone. We find that the natural training objectives fail to ground the audio because their gradient reaches the projector only through attention that has already dismissed it. A connectionist temporal classification loss applied through the frozen output head breaks this deadlock. The resulting model reaches 6.6 percent word error rate on LibriSpeech test-clean, transcribes in roughly eight parallel steps regardless of utterance length, and uses a single adapter trained on six languages, which we evaluate here on English, Hindi, and Mandarin.