Latest

Every published item, in order · 7652 items
Oct 30
PapersSelected10:03
ParaTTT Parallelizes Deep Test-Time Training Memory Layers

The core insight is that deep nonlinear test-time memories no longer have to trade expressiveness for parallelism, since PTTT recovers exact online gradient descent as a fixed point while running chunkwise in parallel. Researchers building subquadratic sequence models should examine the algorithm and benchmarks, since it directly compares against Gated DeltaNet and Mamba-2 at matched model sizes. The paper appears on arXiv as 2610.38255v1.

arXiv cs.LG@Noel Loo et al. (6)Source
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
PapersBuildersSelected09:50
TIDE Speeds FSDP2 Training with Zero-Copy Collectives on 64 GPUs

The hidden cost in FSDP training is often layout conversion and collective interference rather than compute, and TIDE shows both can be largely eliminated with standard NCCL APIs rather than custom hardware. Teams running large-model training on multi-node clusters can adopt it to reclaim step time and GPU memory without changing FSDP2's per-parameter kernels. The mixed-precision bf16-to-fp32 reduce-scatter is a capability stock NCCL lacks, making the implementation details directly reusable.

arXiv cs.LG@Zejian Wang et al. (14)Source
Oct 28

Only the latest 2000 items are listed; for older ones see the daily archive