Vulnerability detection and monitoring using LLM

Abstract

Large Language Models (LLMs) have evolved as a cornerstone for intricate code evaluations in the modern realm of artificial intelligence and machine learning. The prioritizing of rigorous security requirements is a crucial requirement for the business in the dynamic and ever-changing world of software development. The current study has used the capabilities of the GPT-3.5- Turbo model to conduct a detailed assessment of various code snippets to find any vulnerabilities. The main objective of the experiment was to introduce continuous monitoring technologies to enhance software security and release control. To obtain reliable results, we used a classification report and a confusion matrix. Out of these validation methods we choose accuracy as an important metric for this validation because in this experiment we need our model to predict the vulnerabilities that are present in the 2740 test cases and we would need our model to focus more on true positives(TP). The ideal goal of this experiment was to predict any kind of vulnerability from the real-world data. Out of all test cases, we were able to have an accuracy of 0.77. This demonstrates the approach's potential efficacy in discovering vulnerabilities. Nonetheless, the study found certain parts that require improvement, emphasizing the importance of continual refinement in the model's methodology to ensure more thorough security assessments. This study lays the groundwork for future research into the use of powerful machine learning models in the assessment of software vulnerabilities. The findings not only highlight the effectiveness of the existing approach but also offer light on prospective future research directions, paving the way for the next generation of models and evaluation techniques.