A deep-learning-based approach for object orientation estimation to solve MatchKey CAPTCHA using a Siamese Neural Network

Abstract

This paper puts forward a deep learning-based approach to tackle MatchKey Advanced CAPTCHA - an unsolved challenge in the field - by unfolding defiance with Data Preparation and Model architecture. CAPTCHAs are crucial to differentiate humans from automated bots. The accuracy of current CAPTCHA-solving systems is frequently affected by inadequately diversified samples and suboptimal preprocessing techniques. In this work, we utilize Playwright for automated data collection from different sources (e.g., GitHub, Microsoft, etc.) and Django for streamlined annotation management to ensure a high-quality, task-specific, and tailored dataset. Core innovation lies in predicting object orientation and angular alignment in CAPTCHA images - an unprecedented task - using a Siamese Neural Network (SNN) adjusted with contrastive loss. The SNN architecture is deemed ideal for the task because of its ability to learn a distance function that can differentiate the relative orientation of paired images. The data preparation was analogously essential because of the significance of curated datasets. Contrasting with generic datasets, which frequently contain irrelevant examples, our dataset is meticulously selected to cover a broad range of orientation challenges, guaranteeing robust training for an entirely new CAPTCHA-solving paradigm. The efficacy of our combined method is demonstrated by the model's 98.83% accuracy in solving orientation-based, previously unsolved MatchKey CAPTCHA. While highlighting the synergy between customized datasets and optimal neural architectures, this study simultaneously unfastens the door for future AI-driven security application solutions. Moreover, the strategy has wider ramifications for boosting cyber-security and user accessibility in digital systems than just solving CAPTCHAs.