Latest

Open-source inference stacks closing the gap with a lab's proprietary serving setup changes the economics of self-hosting large MoE models. Teams weighing DeepSeek's API against running their own GPUs can use these Wide-EP and EPLB results to estimate revenue per GPU at off-peak prices. The GB200 and GB300 benchmarks also give infrastructure planners concrete throughput targets for the latest NVIDIA hardware.

官方账号vLLM@vllm_project7 sourcesSource

The same serving stack now optimises both NVIDIA and AMD paths, giving operators a concrete reason to benchmark MI355X deployments against GB300 rather than defaulting to CUDA. Teams running agentic inference traces can enable the mode through SGLang and expect P90 latency gains without changing models. The plugin system also means third-party kernels — starting with GLM-5.3, K3, and MiniMax M3 — will arrive without waiting for a core release.

官方账号LMSYS Org (SGLang)@lmsysorg11 sourcesSource

This release significantly improves inference efficiency through advanced attention mechanisms and memory management, enabling faster processing of large models. Developers working with LLMs in production environments will benefit from the optimized serving capabilities and experimental features like engine snapshots, particularly for models requiring high-throughput processing.

官方账号vLLM@vllm_project7 sourcesSource

Multi-agent collaboration is moving from separate orchestration tools into the editor itself, with task status surfaced directly in the UI. Developers running agent teams in daily coding workflows can track members without leaving the chat. Comparing output against V4.1 and Astra results shows how much agent-team quality varies by underlying model.

Teortaxes@teortaxesTex11 sourcesSource