Duopra 论文:让全双工语音模型在对话之外学会执行任务
Duopra: From Full-Duplex Conversation to Full-Duplex Agency
语音智能体卡在对话流畅但任务做不成,这篇论文把工具调用直接塞进全双工模型,Pass@1 从 4.7% 拉到 44.7%。用 DeepSeek 当后端还能到 49.3%。做实时语音 Agent 的人值得照着三阶段训练配方试一遍。
arXiv 论文 Duopra 针对 Full-Duplex-Bench v3(FDB v3)上流式语音智能体的失败模式提出改进:Realtime-Venus-Audio + Harness 管线中大多数失败发生在领域工具执行之前,即任务发起环节出错。Duopra 将后端当作一个接受自包含自然语言请求的工具,让直接调用与后端委派共享同一动作接口,从而把对话控制(打断、插话、接话)与任务执行(及时调用工具、根据用户纠正修改参数)训练在同一模型里。训练分三阶段:协议 SFT、以及时且完整执行为奖励的全双工 RL、以及 thinker 更新后仅对 projector 做的 talker 条件修复。
效果上,在同一后端下 Pass@1 从 4.7% 提升到 44.7%;Duopra 直接调用工具达到 47.7%(SFT 后为 26.0%),配合 DeepSeek 后端达到 49.3%。相比基座模型,它在 FDB v1.0 的三项接管率指标上均有改善。
这项工作表明任务代理能力可以与对话代理能力共存于一个全双工模型中,为实时语音智能体从对话走向可执行动作提供了可复现的训练配方,对构建低延迟语音助手的开发者有直接参考价值。
Duopra: From Full-Duplex Conversation to Full-Duplex Agency
Full-duplex interaction models exhibit strong conversational agency, enabling natural, low-latency turn-taking, backchannels, and interruptions. However, completing tasks further requires task agency: timely tool invocation and argument revision following user corrections. Frontend-backend agents separate initiation from execution: the frontend decides whether and what to request, while the backend determines how. Execution can be offloaded, but unsent requests cannot be executed, and incomplete ones can misdirect the backend. On Full-Duplex-Bench v3 (FDB v3), most failures of Realtime-Venus-Audio + Harness in our streaming pipeline occur before a domain tool executes. We introduce Duopra, a full-duplex interaction model that learns task execution alongside conversational control. Duopra treats the backend as a tool, accepting a self-contained natural-language request. Direct calls and backend delegation share an action interface and requirements for faithful initiation and correct execution. Its three-stage training combines protocol SFT, full-duplex RL rewarding timely and complete execution, and projector-only repair of talker conditioning after thinker updates. With the same backend, Pass@1 rises from 4.7% to 44.7%; Duopra achieves 47.7% calling tools directly (26.0% after SFT) and 49.3% with a DeepSeek backend. It improves three FDB v1.0 takeover rates over the base model. These results show that task agency can coexist with key conversational agency.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Duopra, DeepSeek, voice interaction, agents.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.