墨尔本实验室 i14labs 推出 Sugar Glider 架构，实现 O(1) 恒定显存推理
墨尔本的 i14labs 搞了个 Sugar Glider 架构，把开源大模型改成固定显存的 O(1) 推理，号称几小时就能转换，还测过 1 万亿参数的模型。做推理部署的人值得关注，长上下文成本可能被大幅压下来。
墨尔本实验室 i14labs 发布 Sugar Glider，一套可实现 O(1) 每 token 推理和固定大小 KV cache 的 LLM 架构。官方称能把现有开源权重前沿模型在数小时内转换为恒定显存推理，测试覆盖到约 1 万亿参数规模的模型。
技术上的核心是：随着上下文增长，每个 token 的显存和计算开销保持不变，不再像标准 Transformer 那样线性扩张 KV cache。这意味着长上下文推理不再被显存上限卡死，部署端可以用固定硬件服务更长的会话。
i14labs 的思路对应一个行业现实：数据中心、GPU 和电力扩建面临电网限制、施工延迟和社区阻力，无法无限扩张。在既有硬件上榨出更多算力，正在成为降低推理成本的必要方向。候补名单已在官网开放。
Take an existing open-weight frontier model and convert it to constant-memory inference in a matter of hours. That's what @i14labs says it can do.
The Melbourne lab launched Sugar Glider today, a family of LLM architectures with O(1) per-token inference and a fixed-size KV cache. Memory and compute per token stay flat as the context grows. i14 reports testing the approach on models up to roughly one trillion parameters.
The industry is pouring enormous sums into data centers, GPUs and power to keep up with demand. That buildout can't keep the same pace forever. Power and grid limits, construction delays and growing pushback against new data centers all slow it down.
So getting more work out of the hardware we already have is turning into a necessity. i14 says its architectures do that while keeping frontier performance, and I think a lot of the next gains will come from exactly this kind of work.
Waitlist is open at https://t.co/JUYidkbNxd
This report is categorized under AITOP's "Models" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Sugar Glider, i14labs, 推理优化, KV cache.
Source attribution: curated from kimmonismus (@kimmonismus).
