Involution meets sparse attention:a federated learning based frontier in medical image classification

Abstract

"Lung diseases are among the most common medical conditions worldwide, affecting millions of people across all age groups. Thus, early detection of lung diseases is essential for improving a patient’s health and outcomes. In recent years, deep learn- ing models have demonstrated remarkable performance in classification of medical images. Despite the development of increasingly robust models with higher classifi- cation accuracy, existing hybrid models are not suitable to be deployed in laboratory computers due to the extensive number of parameters they require during the train- ing phase. For example: attention based SWIN and depthwise convolution based MobileNet V2 S have achieved 66.58% and 86.82% accuracy with 29M and 4.5M parameters respectively. Several hybrid models like LeViT, CVT and MobileViT S offer 77.03%, 73.63% and 87.09% accuracies with 7.8M, 19M and 5.6M parameters respectively. These state-of-the-art models are not sufficient to work in a resource constrained environment of hospital laboratories. Moreover, these heavy models need large datasets to achieve generalization. However, acquiring such large-scale datasets in the medical domain remains challenging due to various constraints. For bridging these gaps in the SOTA models, in this research we introduce a light-weight deep learning based hybrid model, InvoSparseNet to compete with the best SOTA hybrid models in the tradeoff between accuracy and parameter count. To train our model, we collected a small, primary CXR dataset consisting of normal, pneumonia and abnormal classes. The aim of InvoSparseNet is to assist medical professionals, such as radiologists, in diagnosing lung diseases from X-ray images and it can be adapted in any medical hospital easily, especially in resource constrained environ- ments. To address this objective,we propose a lightweight architecture based on Involution, Sparse Attention, and depth wise convolution for classifying lung dis- eases. The model is optimized for deployment on PC’s as it can be trained on small datasets because it has less parameters, ensuring accessibility and ease of use. To address challenges in collecting and analyzing medical images due to privacy concerns and the risk of information leaks, we have incorporated federated learn- ing. Federated learning allows multiple hospitals or institutions to collaboratively train the model without sharing raw patient data, preserving privacy while main- taining high performance and data security. With just 3.2 million parameters, our model achieves 86% accuracy on the private dataset, setting a new benchmark for lightweight and high-performance solutions."