Guided Vision: A High Efficient And Low Latent Mobile App For Visually Impaired

Abstract

This paper presents a novel solution for visually impaired individuals. A mobile app is connected to an ESP32CAM and a remote server to help visually impaired individuals to navigate around their environment safely. A deep learning model is deployed in the mobile app to detect obstacles in real-time without connecting to the internet. Other tasks such as reading texts, recognizing people, and describing objects are done in the remote server. We managed to connect the mobile app to the ESP32CAM and the remote server simultaneously. This was possible because the ESP32CAM is connected to the mobile app through Bluetooth. This gave the mobile the ability to connect to the remote server via the internet. To the best of our knowledge, no research has been done using Bluetooth to stream images to do object detection in a mobile app locally. Hence, our solution can detect obstacles locally and do other tasks mentioned previously in the remote server. This paper discusses how the ESP32CAM, obstacle detection module, face recognition module, text reading module, and object description module was implemented such that a low latent and highly efficient mobile app is created using minimal resources.