Latest

Open-source inference stacks closing the gap with a lab's proprietary serving setup changes the economics of self-hosting large MoE models. Teams weighing DeepSeek's API against running their own GPUs can use these Wide-EP and EPLB results to estimate revenue per GPU at off-peak prices. The GB200 and GB300 benchmarks also give infrastructure planners concrete throughput targets for the latest NVIDIA hardware.

官方账号vLLM@vllm_project7 sourcesSource

The same serving stack now optimises both NVIDIA and AMD paths, giving operators a concrete reason to benchmark MI355X deployments against GB300 rather than defaulting to CUDA. Teams running agentic inference traces can enable the mode through SGLang and expect P90 latency gains without changing models. The plugin system also means third-party kernels — starting with GLM-5.3, K3, and MiniMax M3 — will arrive without waiting for a core release.

官方账号LMSYS Org (SGLang)@lmsysorg11 sourcesSource

Hy Memory treats memory as three distinct types—chat events, ongoing work and code—so agents can resume long tasks with the right context instead of raw chat logs. The 39.3% token reduction also means lower inference costs for memory-heavy agent deployments. Developers building agents on LangChain-style stacks can compare it directly against Mem0 and MemOS, both of which it outperforms on the cited benchmarks.

This benchmark result demonstrates the practical effectiveness of combining enterprise frameworks with specialized models for coding assistance. Developers working on complex software engineering tasks can leverage this combination as a verified reference point for building their own coding assistants. The performance compares favorably to other coding assistants in the market.

官方一手pandaily@[email protected] (Pandaily)6 sourcesSource

This release significantly improves inference efficiency through advanced attention mechanisms and memory management, enabling faster processing of large models. Developers working with LLMs in production environments will benefit from the optimized serving capabilities and experimental features like engine snapshots, particularly for models requiring high-throughput processing.

官方账号vLLM@vllm_project7 sourcesSource

Permanently quadrupling the credit signals that DeepSeek's flash-tier economics are now viable for third-party platforms to subsidise, a shift in how cheap frontier-adjacent inference has become. Developers comparing coding-assistant plans can treat this as a direct price benchmark against comparable Claude or GPT flash-tier offerings. DeepSeek's official partnership with OpenCode also shows Chinese labs distributing through agentic coding tools rather than APIs alone.