#recurrent neural networks

9 stories · 0 in 7 days · 3 in 30 days

open-source models 2 AIME 1 AL-RNN 1 Apple 1 DreamerV3 1 Huginn 1 KV cache 1 LoopCD 1

Stories, products and papers tagged "recurrent neural networks", newest first.

Nov 6

Papers10:26

Chinese content截断BPTT的梯度边界效应被证明呈指数衰减

搞RNN或循环模型训练的可以看看，这篇把截断BPTT到底丢多少梯度信息给出了数学刻画，结论挺实用。

#TBPTT #gradient descent #recurrent neural networks #optimization theory

arXiv cs.LG@Julian D. Schiller, Matthias A. MüllerSource

Nov 3

Papers BuildersSelected00:44

Meta and MIT paper shows looping transformer layers boosts efficiency

Looped layers offer a way to trade compute for depth without inflating parameter counts, which matters for teams deploying models under strict memory or latency budgets. Practitioners building small or edge-deployed models can use the paper's findings on loop placement and lightweight adapters to squeeze more quality from a fixed architecture. Compared with simply doubling depth at 51.4% more parameters, adapters at 1.1 to 4.3% are a far cheaper path to gains.

#Meta #MIT #Transformer #recurrent neural networks

rohanpaul_ai@rohanpaul_aiSource

Oct 27

Papers09:39

Chinese content100 个认知任务联合训练：研究 RNN 中的多任务相互作用

用 AL-RNN 在 100 个认知任务上同时训练，解释了任务间什么时候互相帮助、什么时候互相打架，做神经计算建模的可以看看。

#AL-RNN #recurrent neural networks #multi-task learning

arXiv cs.AI@Sílvia García-Hernández et al. (3)Source

Oct 6

Papers BuildersSelected13:42

Fixed-Point Training Speeds Up Looped Language Models

The key insight is that near-fixed-point recurrent states make much of the compute spent on full backpropagation and full KV caches redundant, cutting costs across training, decoding and RL at once. Practitioners building efficient inference stacks or RL pipelines for recurrent models can apply KV sharing and distilled prefill directly. Compared with Huginn's fixed prior, the learned prior plus orthogonal injection improves perplexity at every scale tested up to 1.6B parameters.

#Huginn #looped model #KV cache #recurrent neural networks

arXiv cs.LG@Benhao Huang et al. (7)Source

Oct 2

Models BuildersSelected13:17

Apple presents LoopCD

LoopCD demonstrates how reducing computational steps without sacrificing performance could make AI models more efficient for deployment on resource-constrained devices. Developers working on model optimization should consider this approach, especially when comparing it to alternatives like distillation or pruning techniques.

#LoopCD #Apple #AIME #recurrent neural networks

Event