AITOPAI热报
PapersSelected

Dan Fu在斯坦福CS336客座讲座：KV缓存、Megakernels与Parcae缩放定律

Official accountTogether AI@togethercompute
Post
View the post
Why we picked it

Dan Fu讲KV缓存和Parcae新缩放定律

Dan Fu在斯坦福CS336课程中讲解了推理时的KV缓存、prefill/decode分离技术，以及大规模推理的架构。

Body

他介绍了Megakernels，通过融合GPU操作实现接近光速的LLM解码。还讨论了Parcae，解释了循环Transformer的扩展问题及其修复方法，并提出了新的缩放定律，暗示现有方法可能未充分利用智能潜力。

Original · Together AI

@realDanFu guest lectured in @percyliang's CS336 at Stanford, check out what he covered:

→ The life of a token: KV cache, prefill/decode disaggregation, and what inference looks like at scale

→ Megakernels: fusing GPU ops to hit near speed-of-light LLM decode

→ Parcae: why loop transformers blow up, how to fix it, and some new scaling laws that suggest we've been leaving intelligence on the table

Catch the recording and a deep dive on Parcae below!

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include KV缓存, Megakernels, Parcae, 缩放定律.

Source attribution: curated from Together AI (@togethercompute).

#KV缓存#Megakernels#Parcae#缩放定律#推理优化

Keep reading

Recent stories with the same tag or section
PapersBuilders10:41
Chinese content论文揭示前缀缓存的智能体服务存在亚稳态：一次突发可致吞吐腰斩70分钟

跑过 vLLM 智能体服务的人应该都见过那种莫名的性能崩塌，这篇论文把它解释清楚了：一次几小时的突发流量就能让命中率掉到 10%，吞吐腰斩还自己好不了。论文给了能用的预算器方案，不用调参，直接把 KV 池撑爆的可能性掐掉。

#vLLM#SGLang#prefix caching#KV缓存
arXiv cs.LG@Wen Huang, Jack LiuSource
PapersBuilders12:36
Chinese contentSemanticKV：面向智能体自回归视频生成的语义对齐 KV 缓存压缩

做长视频生成或智能体视频合成的开发者可以看看这篇。它不用重训练，靠规划器的语义约束决定缓存留什么，60 秒生成时内存砍到三分之一，画质反而比不压缩还高一点。四个骨干、三个基准都有数据，方法可直接套在现有 rollout 流程里。

#SemanticKV#视频生成#KV缓存#推理优化
Event
arXiv cs.LG@Weizhong Huang et al. (5)Source
PapersBuildersMulti-sourceSelected13:09
BitDecoding-FP4 Cuts KV Cache to FP4 With 2.4-3.0x Throughput Gains

The co-design removes the trade-off that previously forced a choice between quantization accuracy and decode speed: system-level changes like transposed softmax eliminate lookup overheads that plagued earlier FP4 KV methods. Teams running long-context inference on NVIDIA Blackwell hardware can apply it directly, since the implementation is open-sourced and works across dense and MoE models with GQA and MLA, and it even holds up in RL training rollouts.

#BitDecoding-FP4#NVFP4#Qwen3#KV cache
Event
arXiv cs.AI@Dayou Du et al. (6)Source