Google工程师讲透了边缘AI的内存瓶颈，用小模型做智能体实际可行，还给了离线语音应用的真实案例。
Google AI Edge团队指出边缘AI的真正瓶颈是RAM而非算力，且手机厂商今年缩减RAM、6GB树莓派涨价2.5倍。2B参数的Gemma经2.9-bit量化后可在树莓派上达到约8 tokens/秒，在高通NPU上足以处理几帧视觉。更小的500M至50M参数模型能覆盖老旧设备，通常需微调而非提示。微调后的Gemma在10个动作中实现超86%的函数调用准确率，配合语音模型可完成语音到函数调用。已有一款离线语音听写应用投产，基于两个sub-billion Gemma模型，无需订阅且能去除口头禅。
🆕 Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google https://t.co/HUkLpHUlaJ Th...
🆕 Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google youtube.com/watch?v=hacEQH… J The constraint on edge AI is not compute, it is RAM, and it is getting worse: phone makers are shipping less of it this year, and a 6GB Raspberry Pi costs 2.5 times what it did at launch. So Cormac Brick's team at Google AI Edge spends its effort making models small enough to fit. A 2 billion parameter Gemma, quantized to 2.9 bits per weight, runs on a Raspberry Pi at about 8 tokens per second and on a Qualcomm NPU fast enough for a few frames of vision a second. Below that sit tiny models, from 500 million parameters down to 50, that reach the older laptops and cheap devices where even a small model will not fit. They usually need fine tuning rather than prompting, but the payoff is real: a fine tuned Gemma turns free text into the right function call across ten actions at over 86% reliability, and putting a speech model in front gives you voice to function calling. One shipped example is an offline voice dictation app with no subscription, built on two sub billion Gemma models that also strip your ums and ahs. 💬 0 🔄 0 ❤️ 3 👀 676 📊 1 ⚡