Teger:缓解过挤压问题,提升时空残差误差传播
Improving Spatio-Temporal Residual Error Propagation by Mitigating Over-Squashing
做时空序列预测的团队终于有了一个能同时处理空间相关性和误差时间累积的通用模块——Teger直接插入现有自回归模型就能提升长程预测可靠性,值得在电力、交通等场景试试。
残差误差传播是循环模型中的根本问题,小预测误差随时间累积会降低长程性能。现有深度时序模型虽能高效参数化时变相关性,但常假设误差时间独立并忽略空间相关。
本文提出Teger,一种结构化不确定性模块,通过离散Forman曲率识别信息瓶颈边并进行空间曲率感知图重连,结合低秩加对角协方差头,利用Woodbury恒等式保持可推断性。Teger与骨干网络无关,在LSTM、Transformer和xLSTM上对四个真实时空数据集一致提升了连续排名概率分数(CRPS),并从理论上证明了曲率感知重连能缓解过挤压、改善谱连通性、降低有效电阻并改进协方差校准边界。
Improving Spatio-Temporal Residual Error Propagation by Mitigating Over-Squashing
Residual error propagation remains a fundamental problem in recurrent models, where small prediction inaccuracies compound over time and degrade long-horizon performance. Accurately modeling the correlation structure of such residuals is critical for reliable uncertainty quantification in probabilistic multivariate timeseries forecasting. While recent time-series deep models efficiently parametrize time-varying contemporaneous correlations, they often assume temporal independence of errors and neglect spatial correlation across the observed network. In this paper, we introduce Teger, a structured uncertainty module that overcomes the spa- tial and temporal limitations of error-correlated autoregressive forecasting. Teger proposes a spatial curvature-aware graph rewiring mechanism explicitly strengthening information-bottleneck edges identified by discrete Forman curvature. The component is integrated into a low-rank-plus-diagonal covariance head, preserving tractable inference via the Woodbury identity. Teger is backbone-agnostic, requiring only the latent state produced by any autoregressive encoder. We provide theoretical evidence of Teger, and experimentally evaluate it on LSTM, Transformer, and xLSTM backbones across four real-world spatio-temporal datasets, showing consistent improvement in Continuous Ranked Probability Score (CRPS). We further provide a formal theoretical analysis connecting curvature-aware rewiring to (i) oversquashing alleviation, (ii) improved spectral connectivity, (iii) reduced effective resistance, and (iv) improved covariance calibration bounds
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include spatiotemporal prediction, uncertainty quantification.
Source attribution: curated from arXiv cs.LG (@Seyed Mohamad Moghadas, Esther Rodrigo Bonet, Bruno Cornelis, Adrian Munteanu).