Papers

AI papers from arXiv and journals, each with a summary and why it matters · 7704 papers
Nov 5
PapersBuildersSelected12:12
Systolic Linear Attention Optimises Gated Delta Attention for Google TPUs

Linear attention layers built for GPU tensor cores waste half of each TPU matrix tile, so porting code without redesign leaves large speedups on the table. Teams pre-training on TPUs can adopt SLA-style key width, gate tiling and kernel fusion to cut training time without changing perplexity. Against KDA on TPU v7x, the 11%-24% training speed gain makes this a concrete alternative for MoE runs.

arXiv cs.LG@Gagik Amirkhanyan, Nuojin ChengSource