这篇论文发现,调好预处理参数,线性模型就能干翻Transformer等复杂模型,不用堆算力。
研究使用Ridge回归作为测试床,在8个标准基准上搜索上下文长度、局部归一化、正则化和数据增强的最优超参数。发现最佳回溯窗口长度与预测时序高度相关,且非单调,幂律指数从ETTm2的+0.46到Exchange和Traffic的-0.19。在大多数数据集-预测时距组合上,优化后的线性模型超越了先前的线性预测器,并超过Transformer、MLP和CNN基线在6/8个基准上的表现。优化超参数还可作为数据诊断工具,揭示大型模型隐式学习的结构。
How Good Can Linear Models Be for Time-Series Forecasting?
Time-series forecasting research has been moving steadily toward larger architectures, from specialized transformers to general-purpose foundation models, on the assumption that capacity is what unlocks accuracy. We take the opposite position: most of the gap can be closed at far lower cost by tuning preprocessing rather than scaling models. We use Ridge regression as the testbed, since it has a closed-form solution and interpretable weights, which let the optimal hyperparameters be read off the search directly. We search over context length, local normalization, regularization, and augmentation on eight standard benchmarks and find three patterns. (1) Optimal lookback is strongly series-specific and often non-monotonic in forecast horizon, with fitted power-law exponents ranging from $+0.46$ on ETTm2 to $-0.19$ on Exchange and Traffic, challenging the convention that longer horizons need longer history. (2) Normalizing over a learned trailing fraction of the context, rather than its entirety, is almost universally preferred. (3) Series within the same dataset often disagree on hyperparameters; the optimal degree of cross-series sharing varies from fully shared to fully per-series. The resulting models beat prior linear forecasters on most dataset-horizon entries and exceed Transformer, MLP, and CNN baselines on six of eight benchmarks. The optimized hyperparameters also serve as a diagnostic on the data itself, revealing structures that larger models absorb silently into their learned parameters.