DramaSR-LRM: 推理模型提升长篇剧集说话人识别

Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas

精选理由

一篇论文用推理模型做说话人识别,还建了个53万条对话的新基准,短话识别效果比现有方法强不少。

AI 摘要

研究者发布了DramaSR-532K基准,包含53.2万条带注释对话和900多个角色,需整合听觉、语言和视觉线索。他们提出DramaSR-LRM方法,基于大型推理模型(LRM),通过多模态工具自主聚合上下文证据。在短话语上,DramaSR-LRM显著优于现有基线,因为声学生物特征在这些场景中不可靠。数据和代码将公开。

原文 · arXiv cs.AI

Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas

Long-form TV dramas present a formidable challenge for comprehensive video understanding, where deciphering complex storyline often relies on \textbf{speaker recognition}, the task of accurately attributing each spoken utterance to its respective character. In this paper, we advance this field through two primary contributions. (1) We introduce \textbf{DramaSR-532K}, a large-scale benchmark comprising 532K annotated dialogue lines across more than 900 unique characters, necessitating the integration of auditory, linguistic, and visual cues for speaker recognition. (2) We propose \textbf{DramaSR-LRM}, a robust approach built upon a large reasoning model (LRM). DramaSR-LRM is designed to autonomously aggregate contextual evidence via multimodal tool-use, synthesizing diverse inputs to achieve high-fidelity attribution. Experimental results demonstrate that DramaSR-LRM significantly outperforms existing baselines, particularly on short utterances where acoustic biometrics are inherently unreliable. \textit{All the data and code will be made publicly available at the project page: https://www.github.com/198808xc/DramaSR-LRM.}