TRACE:双人对话中时间感知情感协调检测框架

TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech

精选理由

想研究语音AI如何感知对话中的情感协调?这篇论文提出了新数据集DyadEE和框架TRACE,准确率高达97%,值得做语音交互的朋友看看。

AI 摘要

论文提出DyadEE数据集,包含真实情感协调对话和通过交换伴侣、情感重合成制造的干扰对话。同时提出TRACE框架,将双人交互建模为基于情感微调Whisper声学嵌入的有序序列,将每个样本视为交互痕迹而非池化话语。在DyadEE上实验表明,融入对话上下文和关系信息可提升检测效果,TRACE达到97.01%的准确率。

原文 · arXiv cs.AI

TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech

With the proliferation of speech AI agents, understanding emotional entrainment in conversational interaction has become increasingly important. Emotional entrainment is shaped by social relationships and conversational context, influencing affective coordination over time. We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions, containing both emotionally entrained conversations and synthetic interactions where entrainment is disrupted through partner swapping and emotion resynthesis. We further propose TRACE, a window-level framework that models dyadic interaction as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations, treating each sample as an interaction trace rather than pooled utterances. Experimental results on DyadEE show that incorporating conversational context and relationship information improves emotional entrainment detection, with TRACE achieving the best accuracy of 97.01%.