这篇论文用简单思路修了GDN-2的三个bug，实测1.3B模型困惑度降了0.18，还省内存和参数，想搞高效注意力的话值得看。
CARVE提出仅擦除关键轴的注意力机制，解决了GDN-2的三个耦合缺陷：记忆盲门控、值轴擦除掩码浪费参数、无法使用WY形式三角形分块求解器。在1.3B参数、100B token训练下，CARVE在WikiText上达到困惑度15.72（比GDN-2低0.18，4.5-sigma效应）。它在9个常识推理基准上领先所有循环基线，并在RULER检索探针上取得SOTA。该方案仅带来0.4%吞吐开销、13%更低峰值内存和19%更少参数。论文还包含六个形式化定理，涵盖记忆容量、Lyapunov稳定性等。
CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention
Recurrent models must forget in order to remember, yet the state of the art decides what to erase without consulting what is stored -- the gate sees only the arriving token, not the memory it is about to modify. This memory-blind gating is one of three coupled defects in the leading delta-rule architecture (GDN-2): the value-axis erase mask wastes parameters at the scale of the value projection, and -- as we prove -- mathematically prevents the WY-form triangular chunk solver that makes recurrent training competitive with Transformers. We introduce CARVE (Content-Aware Recurrent with Value Efficiency), which resolves all three problems through one principle: erase only on the key axis. This is provably necessary and sufficient for the WY-form solver to remain valid. Within it, CARVE reuses the recurrent output tensor -- already written to GPU memory -- as a free content signal for the erase gate, and replaces the per-value write-gate projection with a single scalar per head. At initialisation CARVE is bit-identical to GDN-2; any quality difference emerges from what the content gate learns. At 1.3B parameters trained on 100B tokens, CARVE achieves WikiText perplexity 15.72 (minus 0.18 vs. GDN-2, a 4.5-sigma effect), leads every recurrent baseline on nine common-sense reasoning benchmarks, and sets state of the art on every RULER retrieval probe -- at 0.4% throughput overhead, 13% lower peak memory, and 19% fewer parameters. Six formal theorems cover memory capacity, Lyapunov stability, gradient flow, expressivity separation, Pareto-optimal chunk size, and hybrid optimality.