Multi-task learning for flood prediction: joint classification and regression using hybrid CNN-BiLSTM networks with feature gate mechanisms

Abstract

One of the most important issues in disaster management, especially in flooding prone areas such as Bangladesh, is flood prediction. In this thesis, a new multi-task learning method, which integrates convolutional neural networks (CNN) with bidirectional long short-term memory (BiLSTM) networks with attention mechanisms and feature gates, is proposed to perform a full-flood prediction. We use a hybrid architecture to conduct the binary flood classification and continuous meteorological index regression on a history of 65 years of Bangladesh weather (1948-2013). We have created a new CNN-BiLSTM hybrid system that operates on 12-month temporal sequence of 21 engineered weather records at 35 meteorological stations. The architecture includes feature gate mechanism of automatic feature selection, station embeddings of spatial context encoding, attention mechanism of temporal focusing, multi-task learning with joint classification and regression tasks, and sophisticated regularization with SMOTE oversampling and focal loss of class imbalance. We carried out a comprehensive comparative study on seven baseline models including classical machine learning (Random Forest, XGBoost, LightGBM), time series and deep learning (CNN, RNN, LSTM, BiLSTM) models. The optimized CNN-BiLSTM hybrid model gave a state-of-the-art performance of 0.6314 ROC-AUC to classify floods, which is a 11-percent improvement over the worst-performing baseline and demonstrates superiority over all other competing models. The model showed strong results in imbalanced data (7.4% positive class rate) and ensured the interpretability of the model using the feature gates and attention weights. The study contributes to the development of flood forecasting techniques by showing that learners in hybrid neural networks can be used to learn intricate weather-flood dependencies, which can be further used to establish a comprehensive meteorological risk forecasting with direct effects on early warning systems in Bangladesh.