Latest

Every published item, in order · 8120 items
Oct 30
PapersBuildersSelected09:50
TIDE Speeds FSDP2 Training with Zero-Copy Collectives on 64 GPUs

The hidden cost in FSDP training is often layout conversion and collective interference rather than compute, and TIDE shows both can be largely eliminated with standard NCCL APIs rather than custom hardware. Teams running large-model training on multi-node clusters can adopt it to reclaim step time and GPU memory without changing FSDP2's per-parameter kernels. The mixed-precision bf16-to-fp32 reduce-scatter is a capability stock NCCL lacks, making the implementation details directly reusable.

arXiv cs.LG@Zejian Wang et al. (14)Source
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
Oct 28
PapersBuildersSelected10:55
SynthAPI Pipeline Derives Scaling Laws for Tool-Use Agents

The non-obvious finding is that data composition, not raw task count, drives tool-use agent performance, giving teams a concrete recipe for allocating synthetic data budgets. Builders training or fine-tuning agents can apply SynthAPI's control axes to prioritize tool and database diversity over volume. Compared with prior synthetic environment generation work, this approach offers a quantitative scaling law rather than ad-hoc data curation.

arXiv cs.LG@Andrew Szot et al. (8)Source
PapersPolicySelected10:50
Theory Explains Why Unsupervised RLVR Collapses on Large Datasets

The paper turns a known failure mode of self-rewarding training into a predictive formula, giving practitioners a way to estimate safe dataset sizes before a run collapses. It also identifies two concrete mitigations, removing the shared coupling direction or adding a marginal-entropy bonus, that were validated on an actual Transformer. Researchers working on verifiable-reward training and anyone training reasoning models without labels will find the scaling laws directly applicable.

arXiv cs.LG@Peize DingSource
PapersMulti-sourceSelected10:47
Study Explains Why DPO Updates Stay Sparse via NTK Theory

The practical payoff is a theoretical licence to merge DPO-aligned checkpoints: sparser, disentangled updates mean less interference when combining model variants. Researchers working on preference alignment or model merging can use the support-pair bounds to reason about how many samples actually shape a fine-tune. Teams choosing between DPO and SFT for multi-task pipelines get a concrete 35× interference reduction figure to weigh.

Event
arXiv: Anthropic@Kensuke Sasaki, Issei SatoSource

Only the latest 2000 items are listed; for older ones see the daily archive