Papers

AI papers from arXiv and journals, each with a summary and why it matters · 7705 papers
Nov 3
PapersSelected15:17
Scaling Theory Links Multi-Epoch Pretraining Overfitting to Ensembling

As datasets saturate, labs will reuse data across epochs, and this work predicts when that overfits and how ensembling shifts the optimal stopping point. Practitioners planning pretraining or distillation budgets get concrete allocation rules rather than intuition. The FineWeb experiments make the theory testable against single-model scaling baselines.

Event
arXiv cs.LG@Zhen Yang et al. (3)2 sourcesSource
PapersSelected14:29
AttentionIndex Speeds Learned Sparse Attention on GLM-5.3 and DeepSeek-V4-Flash

The method cuts indexer cost, which accounts for over half of prefill time at one million tokens, without the recall degradation that two-stage block approaches like HiSA suffer as context grows. Serving teams running long-context inference on models with learned sparse attention, such as GLM-5 or DeepSeek-V4, can use it to raise prefill throughput 1.3-1.8 times while preserving retrieval accuracy. Both target models come from Chinese labs, making it directly relevant to practitioners deploying these open-weight models.

arXiv: DeepSeek@Jeongmin Brian Park, Wen-mei HwuSource
PapersBuildersSelected14:29
J-PACT Detects Collusion Between LLM Agents via J-Space

Unsupervised detection matters because supervised probes need labelled collusion data that real deployments rarely have, so J-PACT lowers the barrier to auditing multi-agent systems. Teams running agent swarms — especially in finance, trading or messaging pipelines where hidden coordination is hard to spot — can use J-space signals as an oversight layer. The method worked across four open-weight model families, suggesting the collusion signature is not model-specific.

arXiv: DeepSeek@Marco Molinari, Gabriele BettineschiSource