ParaTTT Parallelizes Deep Test-Time Training Memory Layers
The core insight is that deep nonlinear test-time memories no longer have to trade expressiveness for parallelism, since PTTT recovers exact online gradient descent as a fixed point while running chunkwise in parallel. Researchers building subquadratic sequence models should examine the algorithm and benchmarks, since it directly compares against Gated DeltaNet and Mamba-2 at matched model sizes. The paper appears on arXiv as 2610.38255v1.