精选理由
AMD买了家叫Taalas的初创，把模型权重直接烧进芯片，跑Llama 3.1-8B能到每用户每秒1.6万token，但一颗芯片只认一个模型。
AMD收购了加拿大初创公司Taalas，后者将模型权重硬编码进推理芯片，以此换取极致速度。其演示芯片运行Llama 3.1-8B时，每用户每秒可处理超过1.6万个token。这种方案让芯片性能极高，但每颗芯片只能运行单一模型，无法灵活切换。据报道，谷歌也在为Gemini探索类似的芯片内固化方案。
原文 · Decoder
AMD acquires Taalas, a startup that bakes AI models directly into silicon
AMD is buying Canadian startup Taalas, which hard-codes model weights directly into inference chips. That makes them extremely fast but locks each chip to a single model. A demo chip hit over 16,000 tokens per second per user running Llama 3.1-8B. Google is reportedly working on a similar approach for Gemini. The article AMD acquires Taalas, a startup that bakes AI models directly into silicon appeared first on The Decoder .