Are Transformers Effective for Time Series Forecasting?

Responding to a wave of Transformer-based solutions for long-term time series forecasting (LTSF), the paper challenges the validity of this research direction. Its argument is conceptual: Transformers are designed to extract semantic correlations among elements in a long sequence, but time series forecasting instead requires modeling temporal relations across an ordered set of continuous points. Although positional encoding and sub-series tokens retain some ordering information, the permutation-invariant nature of self-attention inevitably causes temporal information loss. To probe this, the authors introduce LTSF-Linear, a family of embarrassingly simple one-layer linear models used as a baseline.

Across nine real-life datasets, LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, frequently by a large margin, and the authors run extensive studies on how various design elements affect temporal relation extraction. The finding matters because it undercuts a popular assumption and encourages the community to revisit whether Transformer-based solutions are truly appropriate for LTSF and other time series analysis tasks such as anomaly detection.