传 DeepSeek Pro 训练因缺少多模态 token 出现模型坍塌
精选理由
有人爆料 DeepSeek 上次 Pro 训练崩了，锅可能是缺多模态数据，官方还没证实，看看怎么说。
一条 X 帖子称 DeepSeek 并非多模态模型，原因是上次 Pro 训练中出现了 model collapse，归因于训练数据中缺少多模态 token。发帖人认为这一诊断若属实，说明 DeepSeek 可以从更早的 checkpoint 恢复训练，补上多模态数据后仍有提升空间。目前该说法未获 DeepSeek 官方确认。
原文 · Teortaxes
> It's not multimodal, as DeepSeek experienced model collapse during the last "Pro" training run, which was attributed to a lack of mm tokens devastating if true That said, if the diagnosis is correct, they can resume from earlier checkpoint, and this suggests it has room to grow
