Transformers_And_LLM_Are_What_You_Dont_Need

By far the best and only repository showing why transformers don’t work in time series forecasting

Star History

Table of Contents

• PhD and MSc Theses

• Videos

• Papers

• Articles

Videos

1. Problems in the current research on forecasting with transformers, foundational models, etc. by Christof Bergmeir

Theses

1. Cotton Price Long-Term Time Series Forecasting: A look at Transformers Suitability

Papers

1. Are Transformers Effective for Time Series Forecasting? by Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu (The Chinese University of Hong Kong, International Digital Economy Academy (IDEA), 2022) code 🔥🔥🔥🔥🔥
2. LLMs and foundational models for time series forecasting: They are not (yet) as good as you may hope by Christoph Bergmeir (2023) 🔥🔥🔥🔥🔥
3. Transformers Are What You Do Not Need by Valeriy Manokhin (2023) 🔥🔥🔥🔥🔥
4. Time Series Foundational Models: Their Role in Anomaly Detection and Prediction (2024) code
5. Deep Learning is What You Do Not Need by Valeriy Manokhin (2022) 🔥🔥🔥🔥🔥
6. Why do Transformers suck at Time Series Forecasting by Devansh (2023)
7. Frequency-domain MLPs are More Effective Learners in Time Series Forecasting by Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Defu Lian, Ning An, Longbing Cao, Zhendong Niu (Bejing Institute of Technology, Tongji University, University of Oxford, Universuty of Technology Sydney, University of Macau, HeFei University of Technology, Macquarie University) (2023) 🔥🔥🔥🔥🔥
8. Forecasting CPI inflation under economic policy and geo-political uncertainties by Shovon Sengupta, Tanujit Chakraborty, Sunny Kumar Singh (Fidelity Investments, Sorbonne University, BITS Pilani Hyderabad). (2024) 🔥🔥🔥🔥🔥
9. Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping by Zhe Li, Shiyi Qi, Yiduo Li, Zenglin Xu (Harbin Institute of Technology, Shenzhen, 2023) code
10. SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction by Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, Qiang Xu (The Chinese University of Hong Kong,2022) code
11. WINNET:TIME SERIES FORECASTING WITH A WINDOW-ENHANCED PERIOD EXTRACTING AND INTERACTING by Wenjie Ou, Dongyue Guo, Zheng Zhang, Zhishuo Zhao, Yi Lin (Sichuan University, China, 2023)
12. A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis by Shuhan Zhong, Sizhe Song, Guanyao Li, Weipeng Zhuo, Yang Liu, S.-H. Gary Chan, The Hong Kong University of Science and Technology Hong Kong, 2023) code 🔥🔥🔥🔥🔥
13. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis by (Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Longj, , Tsinghua University, 2023) code 🔥🔥🔥🔥🔥
14. MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing code 🔥🔥🔥🔥🔥
15. Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift by Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, Jaegul Choo (Kaist AI, Vuno, Naver Corp, ETRI, ICLR 2022) code project page 🔥🔥🔥🔥🔥
16. WINNet: Wavelet-inspired Invertible Network for Image Denoising by Wenjie Ou, Dongyue Guo, Zheng Zhang, Zhishuo Zhao, Yi Lin (College of Computer Science, Sichuan University, China) code 🔥🔥🔥🔥🔥
17. Mlinear: Rethink the Linear Model for Time-series Forecasting Wei Li, Xiangxu Meng, Chuhao Chen and Jianing Chen (Harbin Engineering University, 2023) 🔥🔥🔥🔥🔥
18. Minimalist Traffic Prediction: Linear Layer Is All You Need by Wenying Duan, Hong Rao, Wei Huang, Xiaoxi He (Nanchang, University, Universify of Macau, 2023)
19. Frequency-domain MLPs are More Effective Learners in Time Series Forecasting by Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Defu Lian, Ning An, Longbing Cao, Zhendong Niu (Beijing Institute of Technology, Tongji University, University of Oxford University of Technology Sydney, University of Macau, USTC, HeFei University of Technology, Macquarie University, 2023) code 🔥🔥🔥🔥🔥
20. AN END-TO-END TIME SERIES MODEL FOR SIMULTANEOUS IMPUTATION AND FORECAST by Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Dzung Phan, Roman Vaculin Jayant Kalagnanam (School of Operations Research and Information Engineering, Cornell University; IBM Research, Thomas J. Watson Research Center, Yorktown Heights, NY, USA, 2023) 🔥🔥🔥🔥🔥
21. Long-term Forecasting with TiDE: Time-series Dense Encoder by Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, Rose Yu (Google Cloud, University of California, San Diego, 2023)
22. TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting by Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, Jayant Kalagnanam (IBM Research, 2023) code code
23. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors by Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long (Tsinghua University, 2023) code 🔥🔥🔥🔥🔥
24. Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective Jiaxi Hu, Yuehong Hu, Wei Chen, Ming Jin, Shirui Pan, Qingsong Wen, Yuxuan Liang (2024) 🔥🔥🔥🔥🔥
25. When and How: Learning Identifiable Latent States for Nonstationary Time Series Forecasting (2024) 🔥🔥🔥🔥🔥
26. Deep Coupling Network For Multivariate Time Series Forecasting (2024)
27. Linear Dynamics-embedded Neural Network for Long-Sequence Modeling by Tongyi Liang and Han-Xiong Li (City University of Hong Kong, 2024).
28. PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations (2024)
29. CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables (2024) 🔥🔥🔥🔥🔥
30. Is Mamba Effective for Time Series Forecasting? code (2024) 🔥🔥🔥🔥🔥
31. STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model (2024)
32. TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting code (2024)🔥🔥🔥🔥🔥
33. FITS: Modeling Time Series with 10k Parameters code (2023)
34. TSLANet: Rethinking Transformers for Time Series Representation Learning code (2024) 🔥🔥🔥🔥🔥
35. WFTNet: Exploiting Global and Local Periodicity in Long-term Time Series Forecasting code (2024) 🔥🔥🔥🔥🔥
36. SiMBA: Simplified Mamba-based Architecture for Vision and Multivariate Time series code (2024) 🔥🔥🔥🔥🔥
37. SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion code (2024) 🔥🔥🔥🔥🔥
38. Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting code (2024) 🔥🔥🔥🔥🔥
39. SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters (2024) 🔥🔥🔥🔥🔥
40. Boosting MLPs with a Coarsening Strategy for Long-Term Time Series Forecasting (2024) 🔥🔥🔥🔥🔥
41. Multi-Scale Dilated Convolution Network for Long-Term Time Series Forecasting (2024)
42. ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis code (ICLR 2024 Spotlight)
43. Adaptive Extraction Network for Multivariate Long Sequence Time-Series Forecasting (2024) 🔥🔥🔥🔥🔥
44. Interpretable Multivariate Time Series Forecasting Using Neural Fourier Transform (2024) 🔥🔥🔥🔥🔥
45. PERIODICITY DECOUPLING FRAMEWORK FOR LONG- TERM SERIES FORECASTING code (2024) 🔥🔥🔥🔥🔥
46. Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Models 🔥🔥🔥🔥🔥 (2024)
47. Time Evidence Fusion Network: Multi-source View in Long-Term Time Series Forecasting code (2024)
48. ATFNet: Adaptive Time-Frequency Ensembled Network for Long-term Time Series Forecasting code (2024) 🔥🔥🔥🔥
49. C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting (2024) 🔥🔥🔥🔥
50. The Power of Minimalism in Long Sequence Time-series Forecasting
51. WindowMixer: Intra-Window and Inter-Window Modeling for Time Series Forecasting
52. xLSTMTime : Long-term Time Series Forecasting With xLSTM code (2024)
53. Not All Frequencies Are Created Equal:Towards a Dynamic Fusion of Frequencies in Time-Series Forecasting (2024) 🔥🔥🔥🔥
54. FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting (2024)
55. Long Input Sequence Network for Long Time Series Forecasting (2024)
56. Time-series Forecasting with Tri-Decomposition Linear-based Modelling and Series-wise Metrics (2024) 🔥🔥🔥🔥
57. An Evaluation of Standard Statistical Models and LLMs on Time Series Forecasting (2024) LLM 🔥🔥🔥🔥
58. Macroeconomic Forecasting with Large Language Models (2024) LLM 🔥🔥🔥🔥
59. Language Models Still Struggle to Zero-shot Reason about Time Series (2024) LLM 🔥🔥🔥🔥
60. KAN4TSF: Are KAN and KAN-based models Effective for Time Series Forecasting? (2024) code
61. Simplified Mamba with Disentangled Dependency Encoding for Long-Term Time Series Forecasting (2024)
62. Transformers are Expressive, But Are They Expressive Enough for Regression? (2024) paper showing transformers cant approximate smooth functions
63. MixLinear: Extreme Low Resource Multivariate Time Series Forecasting with 0.1K Parameters
64. MMFNet: Multi-Scale Frequency Masking Neural Network for Multivariate Time Series Forecasting
65. Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysis code
66. CMMamba: channel mixing Mamba for time series forecasting
67. EffiCANet: Efficient Time Series Forecasting with Convolutional Attention
68. Curse of Attention: A Kernel-Based Perspective for Why Transformers Fail to Generalize on Time Series Forecasting and Beyond
69. CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns code
70. Are Language Models Actually Useful for Time Series Forecasting?
71. SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion code
72. FTLinear: MLP based on Fourier Transform for Multivariate Time-series Forecasting
73. WPMixer: Efficient Multi-Resolution Mixing for Long-Term Time Series Forecasting code
74. Zero Shot Time Series Forecasting Using Kolmogorov Arnold Networks
75. BEAT: Balanced Frequency Adaptive Tuning for Long-Term Time-Series Forecasting (2025) 🔥🔥🔥🔥🔥
76. A Multi-Task Learning Approach to Linear Multivariate Forecasting (2025)
77. Benchmarking Time Series Forecasting Models: From Statistical Techniques to Foundation Models in Real-World Applications (2025)
78. Day-ahead demand response potential prediction in residential buildings with HITSKAN: A fusion of Kolmogorov-Arnold networks and N-HiTS (2025)
79. Do We Really Need Deep Learning Models for Time Series Forecasting? (2021)
80. Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning (2023)
81. Are Self-Attentions Effective for Time Series Forecasting? (2024)
82. What Matters in Transformers? Not All Attention is Needed (2024)
83. Time Series Foundational Models: Their Role in Anomaly Detection and Prediction (2024)
84. Performance of Zero-Shot Time Series Foundation Models on Cloud Data (2025) 🔥🔥🔥🔥🔥
85. Position: There are no Champions in Long-Term Time Series Forecasting (2025)
86. FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecasting
87. Cherry-Picking in Time Series Forecasting: How to Select Datasets to Make Your Model Shine
88. TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods 🔥🔥🔥🔥🔥
89. DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting
90. CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns code
91. Can LLMs Understand Time Series Anomalies? code
92. EMForecaster: A Deep Learning Framework for Time Series Forecasting in Wireless Networks with Distribution-Free Uncertainty Quantification
93. Times2D: Multi-Period Decomposition and Derivative Mapping for General Time Series Forecasting code
94. FilterTS: Comprehensive Frequency Filtering for Multivariate Time Series Forecasting
95. OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
96. Does Scaling Law Apply in Time Series Forecasting?
97. TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning code article podcast 🔥🔥🔥🔥🔥
98. FreDF: Learning to Forecast in the Frequency Domain code
99. KARMA: A Multilevel Decomposition Hybrid Mamba Framework for Multivariate Long-Term Time Series Forecasting code
100. MS-TVNet:A Long-Term Time Series Prediction Method Based on Multi-Scale Dynamic Convolution
101. Variational Mode Decomposition and Linear Embeddings are What You Need For Time-Series Forecasting code 🔥🔥🔥🔥🔥
102. TimeSieve: Extracting Temporal Dynamics through Information Bottlenecks code
103. VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Visual Backbones code VisionTS 🔥🔥🔥🔥🔥
104. VISIONTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters code VisionTS 🔥🔥🔥🔥🔥
105. Wavelet Mixture of Experts for Time Series Forecasting
106. TFKAN: Time-Frequency KAN for Long-Term Time Series Forecasting
107. Why Do Transformers Fail to Forecast Time Series In-Context? 🔥🔥🔥🔥🔥 code
108. In-Context Learning of Linear Dynamical Systems with Transformers: Error Bounds and Depth-Separation
109. Time Series Foundation Models: Benchmarking Challenges and Requirements 🔥🔥🔥🔥🔥
110. DoFlow: Causal Generative Flows for Interventional and Counterfactual Time-Series Prediction 🔥🔥🔥🔥🔥
111. Hydra: Dual Exponentiated Memory for Multivariate Time Series Analysis 🔥🔥🔥🔥🔥
112. Mixture-of-KAN for Multivariate Time Series Forecasting 🔥🔥🔥🔥🔥
113. A Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models paper showing foundational models systematically underpeform ARIMA and simple stats ensemble 🔥🔥🔥🔥🔥
114. XLinear: A Lightweight and Accurate MLP-Based Model for Long-Term Time Series Forecasting with Exogenous Inputs
115. Position: The Inevitable End of One-Architecture-Fits-All-Domains in Time Series Forecasting
116. Assessing Electricity Demand Forecasting with Exogenous Data in Time Series Foundation Models

Articles

1. TimeGPT vs TiDE: Is Zero-Shot Inference the Future of Forecasting or Just Hype? (https://towardsdatascience.com/timegpt-vs-tide-is-zero-shot-inference-the-future-of-forecasting-or-just-hype-9063bdbe0b76) by Luís Roque and Rafael Guedes. (2024)🔥🔥🔥🔥🔥
2. TimeGPT-1, discussion on Hacker News (2023)
3. TimeGPT : The first Generative Pretrained Transformer for Time-Series Forecasting