PapersSelected

New AI model for evaluating speech synthesis

Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

Why we picked it

This research introduces a new AI model for evaluating speech synthesis, which could be useful for developers working on TTS systems.

Researchers introduced Live-ProsodyJudge (LPJ), a cost-effective evaluator for live streaming speech synthesis, distilled from Gemini into Qwen3-Omni.

Body

They identified a flaw in standard multi-dimensional evaluation called verdict coupling, where individual dimension scores are aligned with an overall preference.

To fix this, they proposed Decoupled-Live-ProsodyJudge (D-LPJ), which eliminates the overall verdict target and uses a novel GRPO strategy.

Evaluated on human-annotated test sets, D-LPJ successfully produces independent, decoupled dimension judgments, and in a Best-of-8 TTS candidate selection tournament, LPJ-selected utterances were in the human top-3 85.

29% of the time.

Original · arXiv cs.AI

Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors fail to capture. While proprietary Large Language Models (LLMs) like Gemini can evaluate these aspects, they are too costly for massive inference and reinforcement learning feedback. To address this, we first introduce Live-ProsodyJudge (LPJ), a cost-effective pairwise evaluator distilled from Gemini into Qwen3-Omni. However, we identify a critical flaw in standard multi-dimensional evaluation: verdict coupling. The judge tends to lazily align all individual dimension scores with its overall preference, collapsing a rich multi-dimensional rubric into a single preference bit. To resolve this, we further propose Decoupled-Live-ProsodyJudge (D-LPJ). D-LPJ eliminates the overall verdict target to prevent blind following, masks uncertain pair-dimensions during Supervised Fine-Tuning(SFT), and introduces a novel span-local GRPO strategy that applies normalized advantages strictly to their corresponding rationale spans. Evaluated on highly curated human-annotated test sets, 10 sample balanced-order LPJ achieves higher point accuracy than a single Gemini call, while D-LPJ successfully produces independent,decoupled dimension judgments. Furthermore, in a Best-of-8 TTS candidate selection tournament, the LPJ-selected utterance falls within the human top-3 in 85.29% of high-confidence cases, demonstrating its efficacy for fine-grained TTS preference optimization.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Live-ProsodyJudge, Qwen3-Omni, Gemini, speech synthesis.

Source attribution: curated from arXiv cs.AI (@Zifan Guan, Longyu Lu, Junan Zhang, Zhizheng Wu, Meiguang Jin, Junfeng Ma).