Operationally robust machine learning for prelaunch missile failure prediction

Abstract

Pre-launch missile strike failure prediction is a critical defense analytics challenge that requires accurate and trustworthy classification methods. This study introduces a leakage-safe ensemble learning framework that integrates frequency and one-hot encoding, temporal scaling, and SMOTE within cross-validation to address class imbalance (77.1% failures, 22.9% successes). Five classifiers-Random Forest, XGBoost, LightGBM, CatBoost, and stacked ensembles-were evaluated under identical conditions. Random Forest achieved the highest F1-score (96.9%), accuracy (95.3%), and calibration (Brier Score =0.082, ECE=0.038), while stacked ensembles provided a marginal improvement in ROC-AUC (97.5%) but weaker F1 performance. Robustness was further validated through paired statistical tests across 5-fold cross-validation, which confirmed that differences were not statistically significant (p>0.05). Random Forest consistently showed advantages in F 1 and calibration, whereas stacking excelled in ranking. These findings demonstrate that tree-based ensembles are robust and effective for noisy, imbalanced defense data, with Random Forest offering the most reliable and interpretable balance for operational deployment.