论文精选

提出新模型解决直播语音合成中多维度音调评估问题

Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

精选理由

这个研究解决了直播语音合成中一个很具体的问题,提出的模型能更精准地评估语音的情感和语调,对做语音合成优化的人应该有用。

为评估直播语音合成的情感、语调等精细音调,传统方法失效。新提出的Live-ProsodyJudge(LPJ)模型从Gemini蒸馏而来,在Qwen3-Omni上实现低成本评估。但标准多维度评估存在缺陷,新模型D-LPJ通过消除整体判断目标、在SFT中掩蔽不确定维度,并引入GRPO策略,成功实现独立、解耦的维度判断。在最佳8选1的TTS候选选择中,LPJ选出的句子85.29%符合人类前三选择。

原文 · arXiv cs.AI

Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors fail to capture. While proprietary Large Language Models (LLMs) like Gemini can evaluate these aspects, they are too costly for massive inference and reinforcement learning feedback. To address this, we first introduce Live-ProsodyJudge (LPJ), a cost-effective pairwise evaluator distilled from Gemini into Qwen3-Omni. However, we identify a critical flaw in standard multi-dimensional evaluation: verdict coupling. The judge tends to lazily align all individual dimension scores with its overall preference, collapsing a rich multi-dimensional rubric into a single preference bit. To resolve this, we further propose Decoupled-Live-ProsodyJudge (D-LPJ). D-LPJ eliminates the overall verdict target to prevent blind following, masks uncertain pair-dimensions during Supervised Fine-Tuning(SFT), and introduces a novel span-local GRPO strategy that applies normalized advantages strictly to their corresponding rationale spans. Evaluated on highly curated human-annotated test sets, 10 sample balanced-order LPJ achieves higher point accuracy than a single Gemini call, while D-LPJ successfully produces independent,decoupled dimension judgments. Furthermore, in a Best-of-8 TTS candidate selection tournament, the LPJ-selected utterance falls within the human top-3 in 85.29% of high-confidence cases, demonstrating its efficacy for fine-grained TTS preference optimization.