做 LLM 部署的开发者终于有了系统课程——量化降内存 + vLLM 处理并发，直接上手就能优化成本，建议点开学。
Andrew Ng 联合 RedHat 推出新课程，教你如何高效服务大语言模型，以低延迟和合理成本处理大量并发用户。课程核心包括量化降低模型内存占用（如 70B 模型权重约 140GB）以及使用 vLLM 的智能内存管理（如 KV 缓存）来提升并发处理能力。学员将学会量化模型并权衡精度、用 vLLM 部署并观察并发效果、以及基准测试以在速度、成本和精度间做决策。课程适合想优化 LLM 部署的开发者，可直接在 deeplearning.ai 上学习。
New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low la...
New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is built with @RedHat and taught by @cedricclyburn . Efficient LLM serving requires efficient memory management. A 70B-parameter model takes ~140 GB just to load the weights. On top of that, every active request needs its own chunk of GPU memory, the KV cache, to store the token context it has built up so far. In this course, you'll learn to reduce a model's memory footprint with quantization and serve it using vLLM, which handles many concurrent requests efficiently through smart memory management. Skills you'll gain: - Quantize a model and measure the accuracy tradeoff - Serve a model with vLLM and watch it handle concurrent requests efficiently - Benchmark your deployment and make informed tradeoffs between speed, cost, and accuracy Join and learn to serve LLMs efficiently: deeplearning.ai/courses/fast-a… Your browser does not support the video tag. 🔗 View on Twitter 💬 26 🔄 17 ❤️ 142 👀 11956 📊 44 ⚡