本地AI推理正在从概念走向实用，做浏览器应用或关注隐私的开发者值得关注WebGPU的潜力，它可能改变LLM的使用模式。
风险投资人Andrew Chen提出，未来相当比例的LLM查询可能通过本地AI模型在浏览器中运行WebGPU完成，无需发送到云端前沿模型。驱动因素包括：大量查询简单如谷歌搜索，本地模型质量快速提升，苹果等消费硬件已能流畅运行Qwen 3.6 35b MoE等模型，隐私需求（健康、金融等），以及浏览器WebGPU免安装、降低计算成本的优势。尽管云端算力持续增长、token成本下降，但本地推理的便利性和隐私性可能催生新的需求。
How soon before a real % of LLM queries are done via local AI models running webGPU in-browser, and ...
How soon before a real % of LLM queries are done via local AI models running webGPU in-browser, and are never sent to the SOTA model in the cloud? Couple things that might drive this: - you don’t need a frontier model for everything. A very large % of LLM queries are simple, google like queries. Easily handled - local models are getting really good, and getting better - a lot of consumer hardware (particular Apple!) can already run good models pretty well. Newish mac laptop running qwen 3.6 35b MoE LLM at great speeds. Hardware is going to get even better in this direction - there’s def some use cases where people will care about privacy. Health, financial, adult stuff etc. - nice part about browser/webGPU is that there’s no install. It’ll just work. And alleviate compute costs Of course the tension for this is that we’re just going to build a shitload of compute in the world, and tokens will get cheaper over time. Yet it seems like the demand is also so crazy that unlocking a bunch of local supply will be worth it too 💬 14 🔄 0 ❤️ 27 👀 1968 📊 15 ⚡