自动驾驶长上下文处理的计算瓶颈终于有了架构兼容的解决方案——COMPACT-VA 无需修改骨干网络即可压缩令牌,做端到端自动驾驶的团队值得关注其 3.3 倍加速效果。
针对自动驾驶中视觉-动作模型处理长时序上下文时计算开销过大的问题,本文提出 COMPACT-VA 框架。它基于条件 VQ-VAE 实现规划对齐的令牌压缩,将扩展上下文压缩为有限表示,同时保留决策关键信息。该方法在训练时利用未来轨迹蒸馏规划意图,推理时从压缩观测中预测意图,并与压缩记忆拼接后输入策略网络进行端到端优化。在动态场景下,COMPACT-VA 在相同令牌预算下成功率提升超 6%,并实现 3.3 倍加速和 2.7 倍内存缩减。
Planning-aligned Token Compression for Long-Context Autonomous Driving
Monolithic vision-action models represent an emerging paradigm in autonomous driving. However, this architecture produces token sequences that quickly exceed real-time computational budgets when encoding extended temporal context for complex interactions. While approaches like linear transformers and external memory try to make the context lightweight, token compression is most compatible with the architecture as it requires no backbone modifications. Yet existing compression adopts rule-based heuristics like temporal decay, decoupled from planning, risking loss of decision-critical information. We propose COMPACT-VA, a planning-aligned working memory framework built on conditional VQ-VAE, compressing extended context into bounded representations. Compression is conditioned on both historical trajectory and a learned planning intent that the posterior encoder distills from future trajectories during training, while the prior encoder learns to predict it from compressed observations. The compressed memory, concatenated with the predicted latent, feeds the policy for end-to-end optimization, planning with retained decision-critical information. We evaluate on high-signal dynamic scenarios where historical context is most critical for behavior correctness (e.g., stop, yield, or proceed), and accordingly design behavioral metrics. Under comparable token budgets, we achieve $>$6% improvement (68.3%) on success rates with consistent gains across metrics. Ablations validate planning-aligned coupling effectiveness. Closed-loop evaluation confirms that COMPACT-VA maintained general driving performance with 3.3* speedup and 2.7* memory reduction over uncompressed processing.