Ostinato speeds CPU-GPU MoE inference for local agents
Most local agent stacks today force a trade-off between snappy prefill and snappy decode; Ostinato shows the trade-off is avoidable by exploiting routing stability rather than brute-force caching. Teams running MoE models like DeepSeek-V4-Flash on consumer GPU workstations can cut per-tool-call latency by roughly half without changing model behaviour. It matters for anyone building self-hosted agent infrastructure where cloud APIs are not an option.