部署多模态模型时延迟高吞吐低？FlashRT 让智能体自动优化，实测延时降 70 倍，吞吐升 3.6 倍，AMD 平台效果尤其猛。
FlashRT 是一个引导编码智能体优化实时多模态应用部署的框架，将简单参考实现转化为多 GPU 部署。在 NVIDIA B200 GPU 上，FlashRT 将视频世界模型和多模态 LLM 的延迟降低达 70 倍，吞吐量提升 2.8 倍。在 AMD MI355X GPU 上，延迟降低持平，吞吐量提升达 3.6 倍。针对 Qwen3-Omni 文本到音频推理，FlashRT 在 AMD MI355X 上比专家 vLLM-Omni 实现减少 65% 响应延迟。
FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.