推荐系统训练时内存不够用？这篇论文告诉你该加批大小还是加负样本，结论很实用。
该论文研究内存受限下推荐系统训练中批大小与负样本数量的分配问题。理论分析表明，在固定内存预算下，增大批大小比增加负样本数量带来更快的收敛速度。作者提出将内存优先分配给更多样本的实用规则，并在MovieLens-20M等四个真实推荐基准上验证。实验显示该配置在相同内存下获得更优的收敛速度和最终推荐质量。
Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Training
Large-scale neural recommender systems are typically trained with a softmax cross-entropy objective over the full item vocabulary. For a typical large number of possible items $K$, the final classification layer dominates memory, requiring $O(nK)$ logits and gradients to materialize for a batch of $n$ examples. Sampled softmax reduces this cost by restricting the objective to only $k \ll K$ candidate negative items, resulting in an $O(nk)$ memory. However, for a fixed budget $B = n k$, it remains unclear whether one should prioritize larger batches or the inclusion of more negative items. We address this question by analyzing sampled-softmax training under a fixed memory constraint. Under standard smoothness and variance assumptions, our theoretical evidence suggests that the fastest convergence arises from an $ n \sim B, k \sim 1$ allocation. So, an actionable rule is to include as many objects as possible given computational constraints. Our theory is supported by controlled synthetic and synthetic and four real sequential recommendation benchmarks, including MovieLens-20M. The suggested configuration achieve faster convergence and better final recommendation quality than imbalanced alternatives within the same memory constraint. These findings provide a theoretical and empirical foundation for configuring memory during the training of recommender systems. Code, reproducibility materials, and all scripts for generating figures are available at https://anonymous.4open.science/r/LimitedMemoryRule-BBFB