License plate detection and recognition system for all types of Bangladeshi vehicles using multi-step deep learning model

Abstract

A robust license plate (LP) detection and recognition system can extract the license plate information from a still image or video of a moving or stationary vehicle. Bangla license plate recognition is a complicated subject of study due to no publicly available dataset and its specific characteristics with over 100 unique classes, including words, letters, and digits. This paper proposes a robust multi-step deep learning system based on You Only Look Once (YOLO) architecture that can extract license plate information from a real-world image. The resulting system localizes license plates using YOLOv4 object detector model, automatically crops the license plates using bounding box coordinates, enhances the extracted license plate image quality using Enhanced Super Resolution Generative Adversarial Networks (ESRGAN), and then recognizes the classes using YOLOv4 without segmenting the characters. Synthetic images have been used to make proposed method capable of recognizing the classes in unfavorable and complicated conditions. A complete two-part dataset named 'Bangla LPDB-A' is created in this study. This dataset includes Bangladeshi vehicle images with manually annotated license plates and cropped license plates with manually annotated words, letters, and digits. The proposed system is tested on this dataset that has achieved mean average precision (mAP) of 98.35% and 98.09% for final detection and recognition model, which has an average prediction time of 23 ms and 35 ms.