A deep learning approach for multi-class bus fitness classification using a modified faster R-CNN model

Abstract

The high rates of development of the public transportation systems have caused the necessity of the creation of a stable, scalable, and automated system to check the vehicles in order to eliminate additional risk to the passengers and reduce the costs of their maintenance. This paper will present a deep learning driven architecture that uses a customized Faster Region Based Convolutional Neural Network (Faster R-CNN) to classify the bus fitness into multiple classes and hence removes the timeconsuming, inaccurate, and subjective task of manually examining structural defects, body states, and missing parts including side mirrors and headlights. In contrast to the traditional Faster R-CNN models which employ the use of the standard region proposal networks (RPN) and fully connected heads, our design features specialized Multi Layer Perceptron (MLP) heads to enhance feature segregation in subtle defect classes. Pre-processing and augmentation strategies also enhance the methodology by providing resistance to noise and change of viewpoint. This paper further extends the Faster R-CNN architecture of vehicle inspection by tackling the domain-specific limitations, such as small objects of defect and high intra class similarity, and class imbalance, by applying MLP-based classification heads and transfer learning with pre-trained weights, and complementing this with anchor refinement to enhance localization performance. Through experimental tests which include the mean average precision and the recall curves and the confusion matrices, significant gains are achieved compared to the baseline models especially with small or partially visible defects. These works include an expansion of object detection algorithms to safety critical applications, demonstration of the usefulness of feature space expansion using multilayer perceptrons, and the future prospects of implementing these algorithms in roadside camera devices and depot inspection systems, thus providing a base to smart transportation systems that can be applied to trucks, trains and aircraft.