这篇论文测试了四个主流实时语音AI处理哭泣、恐惧、讽刺时的表现，结果它们能听出情绪却不当回事，甚至误判口音和年龄。用语音AI处理敏感场景要小心。
该项研究评估了四个主流实时语音系统——OpenAI GPT Realtime 2、Google Gemini 3.1 Flash Live、Alibaba Qwen3.5 Omni Plus和Omni Flash——在三个需要理解语音语调的场景中的表现。面对哭泣的呼叫者说“没事”、恐惧语气下达的电汇指令、以及明显讽刺的同意，所有系统都依据文字而非语调执行操作。其中三个系统在直接询问时能准确识别悲伤、恐惧或讽刺，但在决策任务中又忽略这些信息。研究还发现系统估计口音和年龄时同样跟随文字偏见而非声学特征。提示词引导仅能部分且不稳定地改善表现。
Real-Time Voice AI Hears but Does Not Listen
Speech conveys information through both words and vocal delivery. We evaluate four leading production realtime voice systems-OpenAI's GPT Realtime 2, Google's Gemini 3.1 Flash Live, and Alibaba's Qwen3.5 Omni Plus and Omni Flash-on tasks where the words and the delivery patterns both convey meaningful information. Across three consequential scenarios, all four systems act on the words rather than the voice. They end calls with crying callers who insist nothing is wrong, approve wire transfers authorized in frightened voices, and enroll callers whose agreement is clearly sarcastic. Surprisingly, this is often not a failure of perception. When asked directly, three of the four systems reliably identify the distress, fear, or sarcasm they later ignore when making decisions. We observe a similar pattern when these realtime voice systems estimate accent and age, as their responses frequently follow the biases of the words rather than the acoustic properties of the speaker. We term this disconnect between perception and action the emotional intelligence gap of voice AI. Prompting systems to explicitly attend to vocal delivery improves performance only partially and inconsistently. Our findings show that current realtime voice AI systems often behave as if speech had been reduced to a transcript, suggesting that they should be used with caution in settings where the tone and emotion of delivery convey important information.