别盲目跑网格搜索了。这篇论文提出Floor-First方法,用资源向量估算性能下限,在DeepSeek-V3.2上验证了不同注意力布局的容量墙差异,帮你理性选部署方案。
Floor-First提出一种基于残差驱动的LLM服务优化工作流,将每个解码步骤建模为五维资源向量(HBM字节、FLOPs、网络字节、网络消息、KV容量),通过求和与取最大值得到乐观下限与悲观下限区间,无需剖析器即可评估重叠质量。以DeepSeek-V3.2风格671B MoE/MLA模型在16块NVIDIA H20 GPU上验证:TP16解码在~74 FLOP/byte下受KV容量限制,仅支持约70并发8K请求;EP16+DP-attention布局将容量墙提升至~644,但单流延迟比TP慢2.4倍。该方法通过资源墙排序而非点基准比较部署方案,且支持新注意力模块的组合式接入。
Think Before You Grid-Search: Floor-First Triage for LLM Serving
LLM serving optimization typically benchmarks many configurations and reaches for heavy profilers when latency targets are missed. We argue for the reverse discipline: estimation is the analytical layer of profiling -- without it, optimization degenerates to grid search. Floor First is a residual-driven triage workflow. Each decode step is modeled as a five-dimensional resource vector (HBM bytes, FLOPs, network bytes, network messages, KV capacity); summing within a resource and maximizing across resources gives an optimistic floor, the plain sum a pessimistic one. Where a measurement lands inside this [max, sum] interval reads out overlap quality before any profiler is opened, and profilers escalate only on residuals above a stated threshold. Deployment alternatives are compared by wall ordering -- which resource wall binds first as load grows -- rather than by point benchmarks. The account is compositional: new attention or state-space variants enter by declaring one module, and the workflow ships as a zero-dependency calculator plus an agent skill that enforces the discipline in agentic optimization loops. As a case study we analyze a DeepSeek-V3.2-style 671B MoE/MLA model on 16 NVIDIA H20 GPUs, whose ridge point of ~74 FLOP/byte (vs ~590 for H100) makes it an extreme decode-oriented part. The floors show TP16 decoding is KV-capacity-limited to ~70 concurrent 8K requests; sparse attention removes the KV-bandwidth term but not the capacity wall; an EP16+DP-attention layout accepts slightly worse same-batch weight traffic for an order-of-magnitude higher capacity wall (~644) -- while single-stream latency favors TP by 2.4x. The layout judgment is thus a computable function of the operating point, explaining why production deployments on identical hardware have shipped opposite attention layouts.