Hugging Face Transformers runs GGUF models at llama.cpp speed
Developers who serve GGUF-quantized open-source models can now run them directly through Transformers with llama.cpp-level speed, removing a common reason to keep two separate inference stacks.
Hugging Face's Julien Chaumond announced a collaboration between the Transformers team and GGML.
Transformers has supported loading GGUF files for a few years, but only by dequantizing them.Thanks to contributor _marcsun, Transformers now uses GGML kernels via the kernels library, reaching the same performance as llama.
cpp.The post credits the GGML team for building the kernels.
transformers x @ggml_org collab 🐐 Lysandre @LysandreJik Transformers has supported loading GGUF files for a few years now, by unquantizing them. Thanks to @_marcsun , we're now using GGML kernels through the `kernels` library to run at the same performance as llama.cpp Huge kudos to the entire @ggml_org for making these kernels! 🔗 View Quoted Tweet📊 1 ⚡
This report is categorized under AITOP's "Products" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Transformers, GGML, GGUF, llama.cpp.
Source attribution: curated from Julien Chaumond (@julien_c). Cross-source corroboration: 3 independent sources have covered this development (including Hugging Face: Blog, arXiv cs.AI), confirming industry-wide signal strength.
