搞 VLM 后训练的人可以看看：Rollplex 把前缀计算塞进解码窗口，32 张 H800 上最高提速 2.24 倍。
Rollplex 是一个针对视觉语言模型强化学习后训练的运行时，它将参考评分和训练阶段分解，并把前缀计算移入 rollout 解码窗口。在 Qwen2.5-VL-32B 上，朴素共置需要约 165 GiB/GPU 显存，Rollplex 通过相位感知内存管理和并行感知权重共享解决显存与张量并行度差异。在 32 张 H800 GPU 上，相比串行共置获得 1.23–1.30 倍加速，相比同 GPU 预算下的分离式部署获得 1.57–2.24 倍加速，同时保持同步 RL 更新语义。
Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Training
Vision-language models (VLMs) enable embodied agents to reason and act from visual observations and language instructions. Reinforcement learning (RL) post-training enhances these capabilities using task feedback, but current on-policy RL runtimes execute rollout, reference scoring, and actor training in strict serial phases. While effective for text-only RL, this phase-granular execution is wasteful for VLMs, where processing dense video inputs and prompt prefixes occupies a large fraction of each phase. Because prefix processing is independent of the generated response, it can be run alongside rollout decoding, which leaves GPU compute capacity underutilized, without breaking synchronous on-policy semantics. We present Rollplex, a runtime that decomposes the reference and training phase and moves the prefix computation into the rollout decode window. Realizing this schedule requires more than concurrent kernel launches: naive colocation of Qwen2.5-VL-32\,B requires roughly 165\,GiB per GPU, while rollout and training prefer different tensor-parallel (TP) degrees and weight layouts. Rollplex addresses these constraints with two mechanisms. Phase-aware memory management controls HBM residency according to producer--consumer lifetimes. Parallelism-aware weight sharing uses the same physical storage for layout-compatible tensors across distinct TP degrees and reconstructs only incompatible tensors, avoiding a complete second actor copy. On 32 H800 GPUs, Rollplex achieves $1.23\times$--$1.30\times$ speedup over serial colocation and $1.57\times$--$2.24\times$ over disaggregation under the same GPU budget, while preserving the synchronous RL update.