Latest

Open-source inference stacks closing the gap with a lab's proprietary serving setup changes the economics of self-hosting large MoE models. Teams weighing DeepSeek's API against running their own GPUs can use these Wide-EP and EPLB results to estimate revenue per GPU at off-peak prices. The GB200 and GB300 benchmarks also give infrastructure planners concrete throughput targets for the latest NVIDIA hardware.

官方账号vLLM@vllm_project7 sourcesSource

The same serving stack now optimises both NVIDIA and AMD paths, giving operators a concrete reason to benchmark MI355X deployments against GB300 rather than defaulting to CUDA. Teams running agentic inference traces can enable the mode through SGLang and expect P90 latency gains without changing models. The plugin system also means third-party kernels — starting with GLM-5.3, K3, and MiniMax M3 — will arrive without waiting for a core release.

官方账号LMSYS Org (SGLang)@lmsysorg11 sourcesSource

Hy Memory treats memory as three distinct types—chat events, ongoing work and code—so agents can resume long tasks with the right context instead of raw chat logs. The 39.3% token reduction also means lower inference costs for memory-heavy agent deployments. Developers building agents on LangChain-style stacks can compare it directly against Mem0 and MemOS, both of which it outperforms on the cited benchmarks.

The design lesson is that tiered model routing beats a single flagship model: classification, routing and field extraction do not need top-tier reasoning, and cheaper Chinese models handle them well enough to cut inference cost by double digits. Engineering leads building internal agents can copy the four principles, especially single-pipeline architecture and shipping features behind default-off flags. It is also a concrete case study of small teams automating sales ops rather than headcount.

shao__meng@shao__meng8 sourcesSource

This benchmark result demonstrates the practical effectiveness of combining enterprise frameworks with specialized models for coding assistance. Developers working on complex software engineering tasks can leverage this combination as a verified reference point for building their own coding assistants. The performance compares favorably to other coding assistants in the market.

官方一手pandaily@[email protected] (Pandaily)6 sourcesSource

This release significantly improves inference efficiency through advanced attention mechanisms and memory management, enabling faster processing of large models. Developers working with LLMs in production environments will benefit from the optimized serving capabilities and experimental features like engine snapshots, particularly for models requiring high-throughput processing.

官方账号vLLM@vllm_project7 sourcesSource

The MXC sandbox layer is cross-platform, so agent isolation patterns built for Windows can be ported to Linux (bubblewrap, LXC) and macOS (Seatbelt) rather than staying Microsoft-only. Developers shipping agents in Southeast Asia can test locally with MAI-Code-1.1-Flash before falling back to cloud models when hardware limits are hit. Teams deploying agents in regulated environments should watch how MXC permission scoping compares with existing container tooling.

Blognone@mk11 sourcesSource

A frontier-class open-weight MoE model with only 25B active parameters means lower inference cost per token, which directly affects how cheaply developers can run agentic coding workloads. Practitioners using Cline can test it against Kimi K3 and DeepSeek V4 Pro at no charge before 13 October and decide whether it earns a paid slot in their stack.

Cline@cline2 sourcesSource