EmbeddingGemma 2 fine-tune achieves SOTA Vietnamese ASR for US$78
Fine-tuning a small slice of an embedding model's audio encoder may be a cheaper route to dialect-scale speech models than full ASR training, since data and compute needs drop to hours instead of weeks. Developers serving underrepresented languages in Southeast Asia can adapt the recipe — partial encoder fine-tuning plus a CTC reader — for other regional languages and accents. Its 6.87% WER edges out paid APIs like Cohere Transcribe and ElevenLabs Scribe v2 on Vietnamese benchmarks.
A community project fine-tuned Google's EmbeddingGemma 2 to build a Vietnamese speech recognition model that tops 8 Vietnamese benchmarks with an average WER of 6.87%, ahead of Gemini-Flash-3.8 (6.92%), Cohere Transcribe (7.01%) and ElevenLabs Scribe v2 (7.08%). Training used 3,000 hours of Vietnamese audio on 8 H100 GPUs for about 3 hours, costing roughly US$78. The method freezes most of the model and fine-tunes only the top 2 of 12 audio-encoder layers, then trains a 6-layer Transformer reader with CTC decoding, avoiding autoregressive token generation for high throughput. The CPU inference runs 200x faster than real-time speech and supports mixed Vietnamese-English speech plus streaming recognition.
谷歌EmbeddingGemma 2的第三个“歪用”用例，只花了78美元训练费用，就做出了越南语ASR的SOTA级模型
3000小时越南语音频训练，8张H100 约3小时 CPU推理速度达实时语音的200倍，支持越英混合口语
大量被通用模型照顾不到的地方语言/口音，可能会迎来一波低成本专用模型
8个越南语基准平均WER为6.87%居首，略超过Gemini-Flash-3.8的6.92%、Cohere Transcribe的7.01%、ElevenLabs Scribe v2的7.08%
网友复用EmbeddingGemma 2的预训练音频编码能力，仅微调12层音频编码器的顶部2层，再训练一个6层 Transformer读取器，通过CTC解码将音频特征映射为文字
这种设计无需自回归逐Token生成，有利于提高转写吞吐量；Reader采用动态音频分块训练，还支持流式识别
以前要为一个小语种或一个地方方言训练高性能STT，数据和算力都是门槛，现在只微调部分编码层，再接上Reader+CTC，就做到了与Gemini同级的识别精度
这条路线如果能跨语言复现，可能会影响大量小语种、方言等细分语音识别市场的竞争格局
来自 @stablequan
#STT #ASR #EmbeddingGemma2
