Ant and Tsinghua release Realtime-Venus, an open realtime multimodal agent
Turning a realtime voice interface into an always-observing agent that keeps working while you talk is a different architectural bet from OpenAI's closed API, and the open weights make it inspectable and self-hostable. Builders of smart glasses, in-car assistants and ambient agents should study the dual-loop split, since offloading slow tool calls to a background harness removes the need to interrupt conversation while waiting. As an open Chinese release it also gives developers an alternative to per-token realtime API pricing.
Ant Group and Tsinghua University have released Realtime-Venus, an open-source alternative to OpenAI's Realtime API for building realtime multimodal agents. The system supports continuous audio-video perception, native full-duplex voice, and asynchronous background agent execution. Its dual-loop architecture splits realtime interaction and long-running tasks into two parallel pipelines, built on two 9B models: Realtime-Venus-Omni for continuous audio-video input and Realtime-Venus-Audio for full-duplex speech, plus a Realtime-Venus-Harness that delegates reasoning, search and tool calls to the background and returns results into the ongoing conversation.
OpenAI Realtime API开源替代方案，蚂蚁和清华搞了一个：Realtime-Venus，支持实时音视频感知、全双工语音、异步Agent执行
一套能持续听、看、说，同时在后台执行复杂任务的实时多模态Agent系统
这对智能眼镜、车载助手来说比较有价值，把实时Agent从一个语音接口变成一个能持续观察环境、随时交流、同时处理任务的助手
双循环架构，它把实时交互和耗时任务拆成两条并行运行的链路，核心由两个9B模型和一个Harness组成
Realtime-Venus-Omni，用来持续感知音视频输入 Realtime-Venus-Audio，原生全双工语音 Realtime-Venus-Harness，把复杂推理、搜索和工具调用委派给后台执行，完事后再把结果带回当前对话
前台持续感知和交流，后台异步执行耗时任务，感知上用户不必为了等待AI完成任务中断对话
#RealtimeVenus #OpenAIRealtimeAPI
