AUNET (Attention-based unified network): leveraging attention based N-BEATS for enhanced univariate time series forecasting

Abstract

This study presents AUNET, an enhanced version of the N-BEATS model specifically designed for univariate time series forecasting by incorporating a multi-head self-attention mechanism. The motivation behind AUNET is to address key limitations of the traditional N-BEATS model, such as redundancy in feature learning, inefficiency in capturing temporal dependencies, and over-complexity for univariate datasets. The proposed model aims to improve the representation of temporal features by selectively focusing on relevant parts of the input sequence, thus enhancing predictive accuracy while maintaining computational efficiency. The AUNET architecture leverages multi-head self-attention layers to capture both short-term fluctuations and long-term dependencies effectively. By integrating attention mechanisms, AUNET dynamically focuses on significant time intervals, minimizing redundancy and improving generalization capabilities. The model’s modular structure allows for an interpretable approach to time series forecasting, providing insights into critical temporal patterns. Experimental results demonstrate that AUNET outperforms the original N-BEATS and other attention-based variations, achieving lower Mean Absolute Error (MAE) 0.8857 and Root Mean Squared Error (RMSE) 0.9896, along with a higher R² score 0.9948, indicating improved prediction accuracy and robustness. Comparisons with models incorporating Neural Attention Memory (NAM), ProbSparse Attention, and Multi-Query Attention further highlight the superiority of AUNET in terms of capturing diverse temporal relationships while balancing model complexity. The findings suggest that AUNET offers a powerful solution for accurate, interpretable, and efficient time series forecasting, particularly applicable in domains such as finance, climate modeling, and energy demand prediction. Future work will explore expanding AUNET’s applicability to multivariate time series and enhancing its interpretability for real-time forecasting applications.