Multi-stage fine-tuning of T5 for low-resource dialects: An AI-driven augmentation framework

Abstract

Dialectal diversity poses significant challenges for Bengali Natural Language Processing, where most existing models are trained primarily on Standard Bangla, resulting in poor performance on regional dialects and amplifying systemic bias. To address this gap, this work presents a large-scale dataset containing 63,303 sentences across 12 Bengali dialects, which highlights substantial class imbalance across regions. This work proposes a multi-stage fine-tuning framework leveraging the T5 model, combined with advanced data augmentation techniques back-translation and paraphrasing and class-weighted training to enhance representation of underrepresented dialects. Experiments conducted on the BanglaDial corpus demonstrate that the proposed method achieves state-of-the-art performance, with T5 reaching 92.4% accuracy, 93.0% recall, and 92.1% F1-score outperforming strong baselines including RoBERTa, BERT, and GPT-NeoX. The results confirm that imbalance-aware optimization and synthetic data generation significantly improve model fairness and robustness, making this work a step forward toward inclusive, dialect-aware NLP systems for Bengali.