做实时语音翻译的团队终于有了低延迟方案——延迟降到1-2秒且翻译质量不降，做会议同传或语音助手的开发者可以直接参考。
传统语音翻译系统依赖语音识别和文本翻译两个独立模块，容易产生级联错误。本文提出一种基于LLM的流式语音翻译架构，让模型不仅能输出翻译结果，还能自主判断是否已听到足够音频来输出。该系统通过输入语音与输出文本的自动对齐进行训练，在多个语言对上的翻译质量接近非流式基线，但延迟仅为1-2秒。这项研究解决了现有SpeechLLM系统无法实时流式输出的痛点，使语音翻译更适用于实时应用场景。
Normally, a system that translates speech into text consists of separate modules for speech recognition and text-to-text translation. Combining those tasks into a SpeechLLM promises to exploit paralinguistic information in the speech and to reduce cascaded errors. But existing SpeechLLM systems are slow since they do not work in a real streaming fashion: they wait for a complete utterance of audio before outputting a translation, or output tokens at fixed intervals, which is not suitable for real applications. This work proposes an LLM-based architecture for real streaming speech-to-text translation. The LLM learns not just to emit output tokens, but also to decide whether it has seen enough audio to do so. The system is trained using automatic alignments of the input speech and the output text. In experiments on different language pairs, the system achieves a translation quality close to the non-streaming baseline, but with a latency of only 1-2 seconds.