#Attention-FFN Disaggregation

1 stories · 0 in 7 days · 0 in 30 days

Stories, products and papers tagged "Attention-FFN Disaggregation", newest first.

Jul 25

ModelsMulti-sourceSelected00:55

Chinese contentvLLM AFD Plugin 发布，实现 MoE 推理 Attention-FFN 分离

vLLM 出了个新插件，把 MoE 推理的注意力层和专家层拆分部署，想分别扩缩容就方便了，支持 NVIDIA 和昇腾卡。

#vLLM AFD Plugin #MoE #Attention-FFN Disaggregation #Ascend NPU

官方账号vLLM@vllm_project4 sourcesSource