Hossain, Muhammad IqbalMaruf, Mahmud Mostofa AlTabassum, MahmudaDisha, Radiah ReazEmon, Salauddin Ahmed2026-08-102026-08-1020252025-10ID 22101132ID 22101151ID 22101545ID 24141192https://hdl.handle.net/10361/28861This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2025.Cataloged from PDF version of thesis.Includes bibliographical references (pages 61-62).Cybersecurity is an issue that is being compounded by the fast development of digital infrastructure as cyberattacks grow more sophisticated and frequent. Vulnerability detection and penetration testing are some of the most important challenges in the area of cybersecurity as they allow organizations to highlight the security vulnerabilities prior to the exploitation. The existing traditional penetration testing is, however, largely manual, expensive, inefficient, and time-consuming, especially in large scale dynamic networks. This paper presents an improved vulnerability detection system using AI that can be used to automate penetration testing to enhance the ability of the system to detect threats more effectively. It incorporates automated attack path exploration, AI based threat intelligence and adversarial defense to increase the precision, versatility and dependability of penetration tests. To make security assessment transient and interpretable, explainable-AI methods are also included. Findings show that AI-based penetration testing also saves considerable time in human work, and it detects high-risk threats more efficiently through the analysis of large datasets, including complicated attack patterns. This piece is emphasizing a change to the conventional way of approaching cybersecurity as it provides a more effective and scalable alternative to the conventional way of doing it.72 pagesen-USAttribution-NonCommercial-NoDerivatives 4.0 InternationalBRAC University theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission.http://creativecommons.org/licenses/by-nc-nd/4.0/CybersecurityCyberattacksulnerability detectionDigital infrastructurePenetration testingAI securityAdversarial defenseSecurity automationVulnerability assessmentCyber threatsThreat detectionDigital threatsMachine learningPenetration testing (Computer security).Computer networks--Security measures--Testing.Computer networks--Security measures.Artificial intelligence.Computer security--Technological innovations.Cyber intelligence (Computer security).Thesis