AI模型精选

Macaron-V1: 混合LoRA后训练使GLM 5.2性能提升,64 GPU训练748B模型

Macaron-V1: How RL Made GLM 5.2 Great Again — MindLab Mixture-of-LoRA Post-Training Pushes Trillion-Parameter Models With 64 GPUs

精选理由

MindLab用Macaron-V1在64块GPU上训出748B模型,比传统RL省资源,适合想低成本强化大模型的人

AI 摘要

MindLab发布Macaron-V1,一种基于混合LoRA的后训练方法,应用于GLM 5.2。该方法包含4个专门的1B参数专家适配器,支持2M token上下文扩展。通过RL训练,748B参数的Venti变体仅用64块GPU即可完成训练,显著降低万亿参数模型的后训练成本。

原文 · Pandaily

Macaron-V1: How RL Made GLM 5.2 Great Again — MindLab Mixture-of-LoRA Post-Training Pushes Trillion-Parameter Models With 64 GPUs

MindLab releases Macaron-V1: Mixture-of-LoRA post-training on GLM 5.2 with 4 specialized 1B-parameter expert adapters, 2M token context extension, and 748B Venti variant trained on just 64 GPUs.