Meta把广告推荐模型训到LLM规模，效率翻倍，MFU到20-25%，算力扩4倍。值得做推荐训练的看看。
Meta的生成式广告推荐模型GEM支撑着Instagram和Facebook的广告推荐，现已在数千块最新GPU上训练。该训练将端到端效率翻倍，MFU达到20-25%。训练FLOPs同时扩展4倍，使广告推荐模型达到LLM规模。
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta’s Generative Ads Recommendation Model (GEM), the foundation model behind ads recommendations across Instagram and Facebook, now trains at LLM scale on several thousand of the latest-generation GPUs. This post goes into the details on how we achieved: doubling end-to-end (E2E) training efficiency to 20–25% Model FLOPs Utilization (MFU) while scaling training FLOPs 4x in [...] Read More... The post GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model appeared first on Engineering at Meta .