序列模型研究者终于有了一个轻量级、可扩展的评估工具，CogScale 能帮你快速筛选架构创新，避免盲目大规模训练的高成本。做记忆与推理能力对比的团队可以直接用这个基准来验证新想法。
CogScale 是一个包含 14 个可扩展合成任务的基准测试，旨在隔离和评估模型在不同参数规模下的认知与记忆能力。它提供了一个轻量级标准化框架，让研究者无需大规模训练即可快速验证架构创新。研究者在严格参数预算（1k、10k、100k）下评估了 GRU、LSTM、xLSTM、ESN、Mamba、Transformer 解码器和编码器-解码器七种架构。结果显示，经典 RNN 和回声状态网络在严格参数预算下擅长基础记忆，但只有注意力机制和现代状态空间模型在推理复杂度和任务难度提升时保持高性能。
CogScale: Scalable Benchmark for Sequence Processing
The ability to maintain and manipulate information over time is a fundamental aspect of living beings and Artificial Intelligence. While modern models have achieved remarkable success in tasks like natural language processing, evaluating the capacity of novel architectures to process sequential information remains computationally expensive and time-consuming. Testing a new architecture often requires scaling up to massive datasets and models, leading to vast computational costs and slow iteration cycles. In this paper, we propose CogScale, a benchmark of 14 scalable synthetic tasks designed to isolate and evaluate specific cognitive and memory abilities at different parametrizable scales. By providing a standardized, lightweight framework, CogScale allows researchers to rapidly validate architectural innovations before committing to large-scale training. To establish a solid baseline, we evaluate seven distinct architectures: Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), xLSTM, Echo State Network (ESN), Mamba, Transformer Decoder, and Transformer Encoder-Decoder. These evaluations are conducted under strict parameter budgets (1k, 10k, and 100k) and across different difficulty levels and scales. Our results show that while classical RNNs and Echo State Networks excel at basic retention within strict parameter budgets, only attention mechanisms and modern state-space models consistently maintain high performance as reasoning complexity and task difficulty scale.