NVIDIA研究团队推出Direct-P方法，解决了Blackwell FP4张量核心在注意力计算中的瓶颈问题。
Blackwell架构的FP4张量核心在注意力计算中面临softmax转换和片上依赖问题。Direct-P方法将分数直接映射为FP4概率，在NVIDIA GB200上达到BF16前向吞吐量的2.13倍。因果路径通过保存量化查询和键重建概率，使用FP8梯度操作数，加速80亿参数单GPU更新达1.14倍。
Hardware-Aware FP4 FlashAttention-4
Blackwell's 4-bit floating-point (FP4) tensor cores do not automatically make attention faster because softmax conversion and on-chip dependencies dominate once its matrix products shrink. We address this with \emph{Direct-P} for noncausal inference and a causal path that passes the forward quantization directly into backward. Direct-P maps scores directly to FP4 probabilities and reaches up to 2.13$\times$ the bfloat16 (BF16) forward throughput on an NVIDIA GB200. The causal path reconstructs probabilities from saved quantized queries and keys and uses 8-bit floating-point (FP8) gradient operands, accelerating a complete single-GPU 8-billion-parameter update by up to 1.14$\times$. Matched distributed training retains FP8 probabilities and values; every tested MXFP4 probability/value training trajectory diverges.