Rhetor: 多智能体排练式直播演示与实时语音问答

Rehearsed Multi-Agent Live Product Demonstrations with Real-Time Voice Question Answering

精选理由

Rhetor能自动生成带实时问答的直播演示,比纯浏览器代理或固定视频强,在Excalidraw上达到100%定位准确率。

AI 摘要

Rhetor是一个多智能体系统,输入运行中的web应用和源码,输出带旁白和实时语音问答的排练直播演示。其跨模态特征表示融合UI探索与源码分析,并采用多策略语义定位器。在四个应用中测试,包括公共白板应用Excalidraw,定位器触发率(sigma-bar)在147个动作中为0.31-1.00。在53个动作的大负载任务上触发率约0.92,Excalidraw在迭代2收敛至1.00。论文还定义了十项指标的基准协议用于评估设计选择。

原文 · arXiv cs.AI

Rehearsed Multi-Agent Live Product Demonstrations with Real-Time Voice Question Answering

Live product demonstrations are a recurring, high-cost activity in software organizations: a human presenter must select features, dispatch the corresponding interactions on a running application, narrate them coherently, and answer questions in real time. Existing automation addresses only fragments -- generalist browser agents target instruction-conditioned task completion, and demo-video tools produce fixed MP4 artifacts that cannot be questioned and silently break under interface drift. We propose Rhetor, a multi-agent system that takes a running web application and its source-code repository as input and produces a rehearsed live demonstration with segment-synchronized narration and real-time voice question answering. The architectural contributions are a cross-modal feature representation that merges UI exploration with source-code analysis into features tagged with discrete focus tiers, a grounded scripter constrained to UI elements observed during exploration and dispatched through multi-strategy semantic locators, a pre-presentation rehearsal loop with explicit convergence and graceful degradation to narration-only segments, and a runtime synchronization invariant that ties each browser action to the audio-end event of its narration segment. Across six pipeline sessions on four deployed applications -- including the public-domain whiteboard application Excalidraw -- the rehearser's internal locator-firing rate (sigma-bar) spans 0.31-1.00 over 147 scripted actions; on the substantial workload (53 actions, full tier differentiation), sigma-bar is approximately 0.92, and on the public-domain reference point the locator-repair step drives convergence to sigma-bar = 1.00 at iteration 2. We additionally define a benchmark protocol of ten metrics across six application categories that would establish, beyond the case study, whether each design choice contributes positively.