A novel approach to efficient multilabel text classification: BERT-federated learning fusion

Sadot A.A.I.M.Maliha Mehjabin M.Mahafuz, Aziz2026-09-232026-09-232023-01-01A. A. I. M. Sadot, M. Maliha Mehjabin and A. Mahafuz, "A Novel Approach to Efficient Multilabel Text Classification: BERT-Federated Learning Fusion," 2023 26th International Conference on Computer and Information Technology (ICCIT), Cox's Bazar, Bangladesh, 2023, pp. 1-6, doi: 10.1109/ICCIT60459.2023.10441264.97983503590152-s2.0-85187388964https://hdl.handle.net/10361/30198Large Language Model (LLM)-based transformers, such as Bidirectional Encoder Representations from Transformers (BERT), are currently gaining significant attention for various Natural Language Processing (NLP) tasks, such as machine translation, classification, and auto-completion. These transformer models demonstrate substantial performance improvements for text classification tasks. Multi-label classification problems often require more computation than binary and multi-class classification problems. Also, the computation requirements become more aggressive if large datasets are considered. Federated Learning (FL) offers a solution to train models in a distributed manner while preserving data privacy. This paper proposes a novel approach for building a machine learning model, which deals with a sizeable textual dataset for multi-label classification leveraging FL. FL has been used to train a compound model constructed by extending Bidirectional Encoder Representations from Transformers (BERT) with a "One-dimensional Convolutional Neural Network (1D CNN)". At first, The experiment was conducted in a single machine (Central) with the entire dataset. Then, the dataset was split into two groups, and the same experiment was performed in a Federated Learning fashion (BERT-FL Fusion). The FL setup considerably reduced the required computing power to derive an equivalent global model while increasing accuracy, precision, and F1 Score and minimizing Hamming Loss.6 Pagesen-USComputational modelingText categorizationBidirectional controlTransformersEncodingConvolutional neural networksTask analysisLinguistic analysis (Linguistics)--Data processing.Linguistics--Technological innovations.A novel approach to efficient multilabel text classification: BERT-federated learning fusionConference Proceeding10.1109/ICCIT60459.2023.10441264