能听能看还能说的虚拟人对话模型引关注
模型边听边看边说，你说话时它能读取你的表情，还能用眼神暗示想插话。比 OpenAI 纯语音模式多了视频和表情输出，做数字人客服或虚拟主播的开发者可以留意这个方向。
一条来自 X 用户 svpino 的分享介绍了同步音视频对话模型：该模型在生成语音和面部表情的同时处理输入的音频与视频流，实现与虚拟人格面对面对话。
核心机制是双向并行——聆听时读取用户的视觉线索，说话时输出带有表情的画面，例如用户能从 AI 的眼神察觉它想插话。这类全双工交互与 OpenAI Advanced Voice Mode、GPT-4o 实时语音等纯音频方案相比，增加了视频输入和表情输出两个维度，让打断、点头等对话细节自然呈现。
对开发者而言，这类模型指向数字人客服、虚拟主播等场景的新交互范式。
This is so cool: A model that lets you have face-to-face conversations with a virtual persona.
The model processes incoming audio and video while it generates speech and facial expressions.
You can talk to the AI persona, and it will use visual cues while listening, making for a much more realistic conversation. For example, you'd see when the AI wants to intervene by looking at it.
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include 多模态, 数字人, 语音交互.
Source attribution: curated from Ate-a-Pi (@svpino).
