7月23日
10:00
10:00官方一手arXiv: DeepSeek@Dongfang Li, Xiaodong Luo, Ruoyu Sun, Xuhui Chen, Linyuan Qiu, Jian Meng, Zhengxuan Lu, Yiting Wang, Yucheng Xie, Tao Guo, Tianxiang Fang, Jing Li, Sihang Chen, Shihao Hong, Chang Liu, Weihua Dai, Zirong Zeng, Ziwei Zhu, Zhuohan Wang, Zhengjun Yue, Igor Vasilyev, Min Liu, Weijian Sun, Xin Chen, Yingmeng Gao, Jinhua Zhou, Taolue Chen, Chenwei Wu, Dong Zhang, Wenlong Jin, Jinmin Xiang, Barkova Maria, Ushakov Anton, Xianfei Jin, Tian Ding, Zhihang Lin, Qian Chen, Linxin Yang, Mingzhe Yang, Bingwei Zhang, Hongzhang Yang, Fangxue Zhang, Shijun Qin, Jie Yu, Cuihua Hu, Tolstykh Vasiliy, Nosov Ivan, Abdullin Amir, Zhichen Zhou, Xin Zhang, Zhixiong Ning, Xutong Zhao, Junjie Huang, Jiajun Liu, Weiyan Kong, Zheng Zhang, Wenhan Luo, Lin Hu, Yangbo Guo, Li Zeng, Shihao Zeng, Baotian Hu, Min Zhang, Haizhou Li, Zhiquan Luo
精选
该论文针对万亿参数MoE模型DeepSeek-V4在Ascend NPU SuperPOD上的全参数后训练,提出分层优化框架,实现34.22%的模型算力利用率(MFU),较开源基线提升2.93倍。基于优化基础设施,论文构建了面向运筹学(OR)任务的连续预训练(CPT)和微调(SFT)工作流。SFT数据集包含10K高质量样本,涵盖4个任务类别和3种问题表示。专用模型DeepSeek-V4-Flash-OR在零样本Pass@1上达到71.81%,比GPT-5.4-Mini高3.98个百分点,比基础DeepSeek-V4-Flash高11.27个百分点。
推荐理由:这篇论文展示了如何在华为Ascend芯片上高效后训练万亿参数模型,并且通过运筹学专用微调,在特定任务上超越了GPT-5.4-Mini。