UniDot:统一序列建模与特征交互的推荐架构

UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation

精选理由

推荐系统工程师看过来,UniDot 把特征交互和序列建模统一了,还在 KDD Cup 拿了亚军,值得研究。

AI 摘要

UniDot 是一种用于大规模推荐系统的新型架构,将特征交互模型与序列模型统一到共享的 token 空间中。它基于因子分解机(FM)视角,利用嵌入内积作为注意力机制的基础,实现两类模型的并行处理。在 TAAC KDD Cup 2026 工业赛道中,UniDot 获得亚军。该架构通过 FM Highway 保留显式点积交互,并采用双稀疏/稠密优化器(Adagrad + Muon)训练。

原文 · arXiv cs.AI

UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation

Industrial recommenders rely on two model families that have evolved largely independently: feature-interaction models over multi-field user/item features, and sequential models over user-behavior histories. Production systems couple them only loosely. To unify the two, we present UniDot, a novel architecture for post-click conversion prediction built from the factorization-machine (FM) point of view: the embedding inner product---which powers collaborative filtering and lets a recommender generalize to unseen user--item pairs---is the same primitive as attention's query dot key scoring, so a single dot-product of tokens can underlie both feature interaction and sequence modeling. UniDot tokenizes non-sequential fields and multi-domain behavioral sequences into one shared token space and stacks a single macro-block in which a token-mixing bus and a sequence-retrieval bus (item tokens cross-attending the histories) run in parallel and exchange state each layer through an MLP-Mixer fusion, while an FM Highway carries explicit per-layer dot-product interactions around the residual stack directly to the classifier. The sequence side is embedded once per forward pass and shared by all consumers, bounding inference latency. Trained with a dual sparse/dense (Adagrad + Muon) optimizer, an auxiliary conversion-delay head, and multi-path mutual learning, UniDot finished as the runner-up on the Industrial track of the TAAC KDD Cup 2026.