AI-driven voice assistance for visually impaired individuals by automating Bengali scene text detection and audio feedback

Abstract

Visually impaired individuals in Bangladesh face significant challenges navigating urban environments independently, where critical information — establishment names, addresses, and directions — is conveyed almost entirely through Bengali signboards. Despite recent advances in assistive technology, no lightweight, end-to-end system existed that could detect Bengali signboards, extract their text, and deliver it as spoken audio in real time. This work presents a five-stage pipeline that addresses this gap: a YOLO26s model detects signboards in street scene images, a second YOLO26s model isolates name and address regions of text interest (RTI), Easy- OCR extracts raw Bengali text, the Gemini API corrects diacritical errors and typographical mistakes in the OCR output, and gTTS synthesizes the corrected text into natural Bengali speech. The system is trained on the publicly available SbNet dataset introduced by Mazumderet al. (2025) [49], achieving 97.14% mAP50 on signboard detection and 97.1% mAP50 on RTI detection — competitive with the state of the art despite using only 40% of the available training data. Qualitative evaluation demonstrates that the Gemini post-processing stage meaningfully improves raw OCR output, correctly resolving address separators, abbreviated place names, and character-level diacritical errors. The pipeline runs on a standard laptop CPU at inference, requiring no dedicated GPU for deployment. This research contributes a novel LLM-based correction layer for Bengali signboard OCR, a data efficiency benchmark for the YOLO26 architecture, and a complete assistive pipeline that extends prior detection-only work to full spoken output.