视频生成团队终于有了解决长序列KV缓存内存爆炸的方案——VideoMLA将内存减少92.7%且不牺牲质量，做长视频扩散的开发者可以直接在B200上试，吞吐量提升1.23倍。
VideoMLA首次将多头潜注意力（MLA）应用于视频扩散模型，通过共享低秩内容潜变量和分离的3D-RoPE位置键，将每个token的KV缓存内存减少92.7%。研究发现，尽管视频注意力并非低秩（99%能量有效秩远超实际潜变量维度），但MLA瓶颈决定了有效秩，而非预训练频谱，从而在压缩比下保持质量。在VBench基准上，VideoMLA在短时视频扩散中匹配基线，在长时任务中取得最佳综合得分，并在单块B200上实现1.23倍吞吐量提升。该工作解决了长序列视频生成中KV缓存内存和延迟瓶颈，为分钟级视频扩散提供了高效方案。
VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion
Long-rollout causal video diffusion has converged on a fixed-size sliding-window KV cache, with recent progress innovating within this layout by changing which tokens occupy the window or how their positions are encoded. The per-head KV layout itself, a dominant contributor to streaming memory and latency, has been mostly left unchanged. In this paper, we present the first study of Multi-Head Latent Attention (MLA) in video diffusion. VideoMLA replaces per-head keys and values with a shared low-rank content latent and a shared decoupled 3D-RoPE positional key, reducing per-token KV memory by 92.7% at every cached layer. We further investigate why MLA succeeds in video diffusion even though the spectral assumption often used to motivate it in language models does not hold: pretrained video attention is not low-rank, with 99%-energy effective rank far above any practical latent dimension. VideoMLA retains quality at compression ratios where direct spectral approximation would predict large reconstruction error. We show that the MLA bottleneck, rather than the pretrained spectrum, determines the effective rank: both spectral and random initialization occupy nearly the full rank budget from initialization, and training preserves this budget while adapting within it. On VBench, VideoMLA matches short-horizon streaming video diffusion baselines, achieves the best overall score at long horizons among evaluated methods, and improves throughput by 1.23x on a single B200.