Latest

Open-source inference stacks closing the gap with a lab's proprietary serving setup changes the economics of self-hosting large MoE models. Teams weighing DeepSeek's API against running their own GPUs can use these Wide-EP and EPLB results to estimate revenue per GPU at off-peak prices. The GB200 and GB300 benchmarks also give infrastructure planners concrete throughput targets for the latest NVIDIA hardware.

官方账号vLLM@vllm_project7 sourcesSource

The same serving stack now optimises both NVIDIA and AMD paths, giving operators a concrete reason to benchmark MI355X deployments against GB300 rather than defaulting to CUDA. Teams running agentic inference traces can enable the mode through SGLang and expect P90 latency gains without changing models. The plugin system also means third-party kernels — starting with GLM-5.3, K3, and MiniMax M3 — will arrive without waiting for a core release.

官方账号LMSYS Org (SGLang)@lmsysorg11 sourcesSource

This benchmark result demonstrates the practical effectiveness of combining enterprise frameworks with specialized models for coding assistance. Developers working on complex software engineering tasks can leverage this combination as a verified reference point for building their own coding assistants. The performance compares favorably to other coding assistants in the market.

官方一手pandaily@[email protected] (Pandaily)6 sourcesSource

This release significantly improves inference efficiency through advanced attention mechanisms and memory management, enabling faster processing of large models. Developers working with LLMs in production environments will benefit from the optimized serving capabilities and experimental features like engine snapshots, particularly for models requiring high-throughput processing.

官方账号vLLM@vllm_project7 sourcesSource

For users who rely on AI coding assistants, WorkBuddy offers a more cost-effective alternative to Codex, especially with models like GLM-5.3-flash that reduce costs by up to 90%.

掘金本周最热@wing9811 sourcesSource