Chinese contentRTX 4070 Ti SUPER 实测：性能提升明显，或有利大模型推理 prefill
有人拿 RTX 4070 Ti SUPER 实测了性能提升，还聊到对 prefill 的影响，自己折腾显卡推理的可以看看数据。
#大模型推理
6 stories · 1 in 7 days · 1 in 30 days
Stories, products and papers tagged "大模型推理", newest first.
Oct 24
Sep 14
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
For developers working with large language models, this paper provides a detailed, technical analysis of GPU utilization on Nvidia's Hopper architecture, which is crucial for optimizing inference performance.
Sep 4
RLVR and OPD Sequential Training Method
This research offers a practical training recipe that improves LLM reasoning performance, with clear switching signals between stages applicable to developers working on reasoning models.
Aug 7
Jul 29
Chinese content量化 Quantization（int4、int8）：如何用更少显存跑大模型
刚学 AI 的赶紧看这条，int4/int8 量化能帮你省下几块显卡的钱，3080 跑 70B 模型不是梦。
Jul 12
Chinese contentRTX Spark真机亮相Bilibili World，CPU与GPU集成，笔记本可跑120B模型
老黄把CPU和GPU焊在一起了，笔记本就能离线跑120B的模型，想玩本地大模型可以关注这个新硬件。