FP4 FlashAttention-4 Hardware Optimization
NVIDIA GB200 users gain 2.13× faster attention inference and 1.14× faster training updates with this optimization.
NVIDIA GB200 users gain 2.13× faster attention inference and 1.14× faster training updates with this optimization.