AMD NPU runs LLaDA-8B-Instruct diffusion model efficiently
Diffusion language models are mostly benchmarked on GPUs, so this shows a practical path to running one locally on consumer NPUs where power budgets matter. The measured 8.9x energy gain over CPU INT4 is the kind of number edge-deployment engineers can use when sizing on-device inference. Anyone working on AMD Ryzen AI hardware or quantized LLM deployment should watch for the full poster at PyTorchCon North America 2026.
AMD engineers Ravi Gupta, Shiksha Patel and Rishi Madduri will present a poster on efficient inference of masked diffusion language models on the AMD XDNA2 NPU at PyTorch Conference North America 2026. Their work deploys LLaDA-8B-Instruct on a Ryzen AI MAX+ 395 APU. At sequence length 128, the NPU vendor path ran at 504 ms per forward pass at 36 W, versus 843 ms and 110 W for CPU INT4 — 1.67x faster and 8.9x more energy-efficient. The poster also covers the INT4 quantization and graph-partitioning pipeline, a CastAvx fix, and the runtime approach for single-step prefill on the XDNA2 NPU.
Ravi Gupta, Shiksha Patel, and Rishi Madduri (@AMD) will present “Efficient Inference of Masked Diffusion Language Models on AMD XDNA2 NPU” at #PyTorchCon North America 2026.
Their poster presents a power-efficient deployment of LLaDA-8B-Instruct on an AMD NPU. On a Ryzen AI MAX+ 395 APU at sequence length 128, the NPU vendor path ran at 504 ms per forward at 36 W, compared with 843 ms and 110 W for CPU INT4, making it 1.67x faster and 8.9x better on energy.
The poster will also cover the INT4 quantization and graph-partitioning pipeline, the CastAvx fix, and the runtime approach used for single-step prefill on the XDNA2 NPU.
Register for PyTorch Conference North America 2026: https://t.co/jBApW8nESi
