ModelsMulti-source

EXO Labs 与 Thinking Machines 让 Inkling 2 Mini 本地跑出 210 tok/s,速度比 Claude Sonnet 5.5 快 1.8 倍

> 1.8x faster than Claude Sonnet 5.5 on an M5 Ultra Mac Studio and 11x cheaper (API pricing). An...

Why we picked it

如果你想在 MacBook 上不联网跑代码模型,这条值得看:Inkling 2 Mini 在 M5 Ultra 上每秒 210 token,做编码任务比 Claude Sonnet 5.5 快 1.8 倍还便宜 11 倍。 looping 架构让小模型本地也能堆推理算力,独立开发者直接受益。

EXO Labs 与 Thinking Machines 合作,把 Inkling 2 Mini 优化到可在 Mac 上高效本地运行:在 96GB M5 Ultra Mac Studio 上解码速度达 210 tok/s、prefill 达 3620 tok/s;在 48GB M5 Max MacBook Pro 上也达到 125 tok/s 解码、1750 tok/s prefill。

Body

实测中,它在一台 M5 Ultra Mac Studio 上构建原生天气应用的速度比 Claude Sonnet 5.5 快 1.8 倍,按 API 定价计算便宜 11 倍。关键在于其 looping 架构:循环复用模型层可以在不增加模型体积的情况下提升推理时算力,因此更适合内存受限的本地设备。

这一结果表明开源模型加上本地推理优化,已在编码任务上对云端闭源模型形成速度与成本的双重优势,对需要在 MacBook 上离线跑模型的开发者是直接可用的方案。

Original · Julien Chaumond

> 1.8x faster than Claude Sonnet 5.5 on an M5 Ultra Mac Studio and 11x cheaper (API pricing). An...

> 1.8x faster than Claude Sonnet 5.5 on an M5 Ultra Mac Studio and 11x cheaper (API pricing). And this, my friends, is how local and open source AI is going to win 🔥🔥🔥 EXO Labs @exolabs We worked with @thinkymachines to make Inkling 2 Mini run fast locally: - 210 tok/sec decode, 3620 tok/sec prefill on 96GB M5 Ultra Mac Studio - 125 tok/sec decode, 1750 tok/sec prefill on 48GB M5 Max MacBook Pro It built a native weather app 1.8x faster than Claude Sonnet 5.5 on an M5 Ultra Mac Studio and 11x cheaper (API pricing). The looping architecture is ideal for local devices like MacBooks, which are memory constrained. Looping layers of the model increases inference-time compute (and intelligence) without increasing the size of the model.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Inkling 2 Mini, EXO Labs, Thinking Machines, Claude.

Source attribution: curated from Julien Chaumond (@julien_c). Cross-source corroboration: 11 independent sources have covered this development (including Decoder, Clement Delangue, Artificial Analysis), confirming industry-wide signal strength.