Hugging Face 把 Transformers 模型直接跑进 vLLM，性能不输手写，以后搞推理不用重复造轮子了。
Hugging Face 宣布 Transformers 模型现在可以直接在 vLLM 推理引擎中运行，达到原生速度，部分场景超越手写实现。此前，新模型架构需要在 Transformers（训练/研究）和 vLLM（生产推理）中分别实现，导致重复工作与维护成本。现在，模型作者只需在 Transformers 中实现一次，即可自动兼容 vLLM 的优化栈。在 4B 到 235B 参数模型的基准测试中，包括张量并行和 MoE 设置，Transformers 后端吞吐量匹配或超过原生 vLLM。
Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at nat...
Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at native speed, often matching or beating hand-written implementations. Until now, every new architecture often needed to be built twice: - Once in Transformers for training and research - Again in vLLM for fast production inference That duplication slowed down new models, added maintenance, and created room for implementations to diverge. Now, model authors can implement an architecture once in Transformers and immediately benefit from vLLM’s optimized inference stack. In our benchmarks, the Transformers backend matched or beat native vLLM throughput across models from 4B to 235B parameters, including tensor parallel and MoE setups. One readable model implementation can now power training, fine-tuning, evaluation, RL rollouts, and production inference. The conventional wisdom is that abstractions make systems slower. The best abstractions make the whole ecosystem faster. Write the model once. Deploy it everywhere. huggingface.co/blog/native-sp… 💬 21 🔄 26 ❤️ 132 👀 13614 📊 46 ⚡