时间序列模型为何需要长上下文窗口?

Why Do Time Series Models Need Long Context Windows?

精选理由

做时间序列预测的团队终于有了理论依据——长窗口不只是为了捕捉长依赖,更是为了降低过程识别的不确定性,建议做预测架构设计的开发者仔细看看。

AI 摘要

这篇论文揭示了长上下文窗口对时间序列预测模型的好处不仅在于捕捉长距离依赖,更在于降低生成过程识别的不确定性。作者将预测任务分解为生成过程识别和条件预测两个目标,证明即使对于记忆长度为P的过程,输入窗口也必须大于P才能达到最小误差。通过解耦这两个目标,可以在不牺牲准确性的前提下提升计算可扩展性。实验在合成和真实数据上验证了这些见解对设计预测架构的指导意义。

原文 · arXiv cs.AI

Why Do Time Series Models Need Long Context Windows?

Modern deep learning models for forecasting groups of time series rely on increasingly longer observation windows. However, the benefit of increasing the window size is often simply attributed to capturing long-range dependencies, and broader discussion on how global forecasting models leverage input observations has been limited. In this paper, we show that forecasting groups of time series involves two objectives: (i) generative process identification (GPI), i.e., inferring the specific process generating the input sequence, and (ii) conditional forecasting (CF), i.e., predicting future values given input observations. From this perspective, optimal predictions can be interpreted as an average over plausible data-generating processes, weighted by their likelihood given the input window. This suggests another explanation for the benefits of long context windows: they reduce the uncertainty about which specific process is generating the input time series during operation. We prove that even for processes with memory length $P$, an input window size strictly larger than $P$ is necessary to achieve the minimum attainable error. Finally, we show how decoupling GPI and CF can improve computational scalability without compromising accuracy. Experiments on synthetic and real-world data validate our insights and their relevance for designing forecasting architectures.