DeepSeek-V4-Flash-0731 成为 Ollama 云默认模型，输出速度 200+ tps

Most local agent stacks today force a trade-off between snappy prefill and snappy decode; Ostinato shows the trade-off is avoidable by exploiting routing stability rather than brute-force caching. Teams running MoE models like DeepSeek-V4-Flash on consumer GPU workstations can cut per-tool-call latency by roughly half without changing model behaviour. It matters for anyone building self-hosted agent infrastructure where cloud APIs are not an option.

The key insight is that prefill memory spikes, not just KV cache size, throttle concurrency — streaming layer histories through reusable buffers removes that admission bottleneck. Engineers running long-context inference on SGLang can adopt the design to serve more users per GPU without changing model weights. Compared with HiSparse and ECHO, the gains come from better indexing rather than more hardware.

Unsupervised detection matters because supervised probes need labelled collusion data that real deployments rarely have, so J-PACT lowers the barrier to auditing multi-agent systems. Teams running agent swarms — especially in finance, trading or messaging pipelines where hidden coordination is hard to spot — can use J-space signals as an oversight layer. The method worked across four open-weight model families, suggesting the collusion signature is not model-specific.

The key insight is that KV blocks are write-once and immutable, so caching hierarchies designed for general workloads actively hurt LLM serving — placing blocks on their final medium at creation outperforms migration-based designs. Engineers building or tuning inference infrastructure should evaluate flat layouts and GPUDirect Storage paths against Mooncake-style tiers, especially for agentic sessions with multi-gigabyte prefixes. The benchmark comparisons against Mooncake and Mooncake-GDS provide concrete baselines for such evaluations.