Modeling serially dependent data: From ARIMA models to transformers

The great advances in sensor technology and automated data collection schemes allow for real-time data collection at high frequencies, often resulting in serial dependence, also known as autocorrelation. The analysis of such data requires specialized models as most standard approaches to modeling and testing cannot properly handle serial dependence. In this Quality Quandaries, we first discuss the classical approach of Autoregressive and Moving Average models from a practical perspective. We then showcase some deep learning counterparts to these models following a similar model evolution. We also present the so-called Transformers, a relatively newer modeling approach and provide a comparative example involving some of the models we cover. Our aim is to offer the readers of this Quality Quandaries a comparative overview of the ever-growing class of models that can handle serially dependent data.