Ant Group and vLLM Open-Source Humming Quantized Inference Kernel Library
Ant Group and vLLM Open-Source Humming, Quantized Inference Kernels With Zhenwu 810E Support
Open kernel-level support for extreme low-bit formats like INT1 and FP4 could make cheap quantized inference viable on commodity accelerators, not just high-end GPUs. Developers running vLLM deployments should watch the Zhenwu 810E integration as a signal of Ant building an in-house hardware stack. It also strengthens vLLM's coverage of Chinese-designed AI chips.
Ant Group (蚂蚁集团) and the vLLM community have jointly open-sourced Humming, a JIT GEMM kernel library for quantized inference. Humming covers quantized formats from INT1 to INT8 and FP4 to FP8. The project provides full support for T-Head's (平头哥) Zhenwu 810E chip. Its goal is to improve operator execution efficiency for quantized models during inference.
Ant Group and vLLM Open-Source Humming, Quantized Inference Kernels With Zhenwu 810E Support
Ant Group and the vLLM community open-sourced Humming, a JIT GEMM kernel library for quantized inference. It covers INT1-8 and FP4-FP8 mixes and fully supports T-Head's Zhenwu 810E.
