Which matters more: Model or language? an empirical study in English-Bangla mental health classification

Abstract

We present an empirical comparison of classical baselines and pretrained transformer encoders for mental health status classification across English and Bangla social media text. Using two public datasets-an English multi-class corpus and a Bangla binary corpus-we evaluate TF-IDF with Logistic Regression and Random Forest against BERT, RoBERTa, DeBERTa, and BanglaBERT under a matched setup with a stratified eighty twenty split and macro F1 for model selection. In English, RoBERTa achieves 81.6% accuracy with a macro F1 of 78.8, while TF-IDF with Logistic Regression reaches 77.3% accuracy. In Bangla, BanglaBERT attains 88.3% accuracy with a macro F 1 of 88.3, and classical baselines surpass several non-Bangla encoders. Findings highlight the value of language models and the importance of classical machine learning models in classifying mental status across different languages.