这篇论文解决了大模型训练资源门槛高的问题——单节点8卡就能训120B模型，做MoE和模型扩展的团队可以直接参考其状态保持原则和TQP策略，省下大量硬件成本。
该论文报告了在单个8GPU节点上端到端训练千亿参数稀疏混合专家模型的方法。LightningLM 0.1V 是一个基于循环骨干的语言模型家族，从小型稠密种子模型逐步扩展至120B参数、460个路由专家（top-12路由）。通过状态保持扩展策略，每个更大模型从小模型的训练权重生长而来，活跃参数从1.78B单调增长至5.93B。关键创新包括：可逆循环栈使激活内存不随模型增长而增加；状态保持扩展原则确保各阶段扩展不失败；单节点经济策略TQP通过量化基专家权重和训练低秩适配器，将优化器状态从100B+压缩至2.26B参数。模型家族、分词器和训练代码均已开源。
Reversible Foundations: Training a 120B Sparse MoE through State-Preserving Scaling
This paper reports on training a hundred-billion-parameter sparse mixture of experts on a single eight-GPU node, end to end. LightningLM 0.1V is a recurrence-backbone language model family grown in four stages from a small dense seed, through a 5B and a 9B mixture of experts, to a 120B model with 460 routed experts under top-12 routing. Each larger model is grown from the trained weights of the smaller one; active parameters rise monotonically from 1.78B at the dense seed to 5.93B at 120B (about 5% of the 118.67B stored). The full lineage runs on single nodes, the larger stages at 8K context, reaching a released training loss of 1.78 at 120B scale. This is a systems and experience report. It is organized around three disciplines. Reversibility: a reversible recurrence stack reconstructs activations in the backward pass instead of storing them, holding activation memory flat as the model grows. State-preserving growth: each expansion (dense to MoE, shallow to deep, few experts to many) is given as a reproducible principle paired with the failure that results from getting it wrong; several failures are silent. Single-node economics: the 120B trains through TQP, a strategy of quantized base expert weights and trained low-rank adapters that carries optimizer state on 2.26B adapter parameters rather than 100B+ resident in routed experts, cutting expert-path optimizer state by a factor of ~45. What is new is the integration of known primitives, not any primitive in isolation: one grown lineage running end to end on a single node, documented at practitioner level, with per-domain held-out loss as evidence that targeted capabilities (multilingual Indic competence, code) were learned by construction. Model family, tokenizer, and training code are released.