VPP解决了长上下文LLM推理中流水线气泡问题，在保持短序列性能的同时显著提升长序列吞吐。
VPP是一种新的虚拟流水线并行技术，用于优化长上下文LLM推理中的分块预填充。该方法保持块大小固定，通过虚拟流水线布局减少流水线气泡。在vLLM-Ascend中实现后，VPP在16块Ascend 910C NPU上测试了三种MoE模型，序列长度达100万token。相比DCPP，VPP在长序列上吞吐量提升13.1%，在混合工作负载上提升6.7%，在512K token的DeepSeek-V3.1预填充任务中，流水线气泡比例从6.4%降至0.1%。
VPP: Virtual Pipeline Parallelism for Efficient Chunked Prefill in Long-Context LLM Inference
Chunked prefill pipeline parallelism (CPP) is a key technique for LLM inference. However, equal-size chunks exhibit imbalanced latency, as later chunks attend longer prefix KV caches and incur higher attention costs, leading to pipeline bubbles. Existing approaches mitigate this imbalance through dynamic chunk resizing (Dynamic CPP, DCPP), but our measurements show that this trades scheduling overhead for load balancing, which becomes unfavorable on long sequences. In this study, we propose Virtual Pipeline Parallelism (VPP), which keeps chunk sizes fixed and optimizes the pipeline layout through virtual stages. A V-shaped virtual-stage traversal overlaps each chunk's expensive middle stages with the lighter head and tail stages of its neighbors, while asynchronous communication and pipelined packing further reduce communication stalls and cross-request drain bubbles. We implement VPP in vLLM-Ascend and evaluate it on three MoE-based LLMs with sequences up to 1M tokens on 16 Ascend 910C NPUs. VPP improves throughput by up to 13.1% over DCPP on long sequences and 6.7% on mixed workloads, while preserving performance on short sequences. On a 512K-token DeepSeek-V3.1 prefill workload, VPP reduces the pipeline bubble ratio from 6.4% to 0.1%, achieving a 98.0% reduction compared with DCPP.