Enhancing diagnostic accuracy in multi-modal medical imaging through targeted data processing and customized ensemble learning

Abstract

Misdiagnosis in medical imaging poses critical patient health risks due to limited cross-validation mechanisms for radiological decisions. This study introduces a structured data-processing workflow and a multi-stage prediction system aimed at improving disease detection accuracy across X-ray, MRI, and CT images. The system employs transfer-learning-based Convolutional Neural Networks, specifically EfficientNetV2S and Inception-ResNetV2, supported by modality-appropriate preprocessing. Histogram equalization, adaptive histogram equalization (AHE), and contrast-limited adaptive histogram equalization (CLAHE) are applied to enhance image consistency, while extensive augmentation - including rotation, Gaussian filtering, scaling, shearing, jittering, flipping, and sharpening - mitigates dataset variability. The ensemble framework combines five models through three decision-fusion strategies: majority voting, selective class-wise voting, and customized weighted voting. These fusion mechanisms reduce model-specific biases and strengthen the stability of predictions across 18 disease classes. Validation using confusion matrices shows that selective class-wise voting achieved 95.27% accuracy, while customized weighted voting achieved 94.07%. Grad-CAM visualization is included to support interpretability of the overall decision process. The findings demonstrate the contribution of structured preprocessing and decision-level fusion to improved diagnostic reliability and highlight the system's potential as a second-opinion tool for radiologists.