Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Autoformer addresses long-term time series forecasting, a critical need for applications such as extreme-weather early warning and long-term energy planning. Prior Transformer-based models rely on self-attention to discover long-range dependencies, but intricate long-term temporal patterns make reliable dependencies hard to find, and the sparse point-wise attention used for efficiency creates an information-utilization bottleneck. Going beyond Transformers, the authors design Autoformer as a decomposition architecture that renovates series decomposition from a pre-processing step into a basic inner block of the deep model, giving it progressive decomposition capacity for complex series.

Inspired by stochastic process theory, Autoformer introduces an Auto-Correlation mechanism based on series periodicity that conducts dependency discovery and representation aggregation at the sub-series level, outperforming self-attention in both efficiency and accuracy. In long-term forecasting, Autoformer achieves state-of-the-art accuracy with a 38% relative improvement across six benchmarks spanning five practical applications: energy, traffic, economics, weather, and disease. The authors release the code publicly.