Object detection under diverse weather conditions for autonomous systems using visual prompting

Abstract

Object detection plays a vital role in enabling autonomous systems to perceive their surroundings and make effective decisions while navigating in complex environments. In recent years, remarkable advancements have been achieved in this field by leveraging pretrained models. However, the object detection process across diverse weather remains a challenge for these pretrained models as different weather conditions introduce visual distortions in the images. To address this problem, we propose a novel approach by combining visual prompting with fine tuning of real time object detection models to improve the detection accuracy. Visual prompting enables lightweight input level adaptation by introducing learnable prompts which adjust the input representation during training time by adding only a small number of parameters. In this research, visual prompting was integrated with the YOLOv8 model and the modified YOLOv8 model with ResNet50 backbone. The models were evaluated on five distinct weather datasets including daytime-clear, daytime-foggy, night-clear, dusk-rainy and night-rainy conditions. where the experimental results demonstrated significant improvement in different weather conditions for both models. Thus this research provides an efficient strategy to improve the perception capabilities of autonomous systems in challenging environments.