Source-linked AI summary
Smart Guiding Glasses for Visually Impaired People in Indoor Environment
Jinqiang Bai, Shiguo Lian, Zhaoxiang Liu, Kai Wang, Dijun Liu
TL;DR
Indoor navigation is difficult for visually impaired people because conventional sensing can miss small or transparent obstacles and users need actionable directional guidance. The paper presents smart guiding glasses that fuse depth and ultrasonic sensing, then provide auditory cues or AR visual enhancement. Experiments report robust obstacle avoidance and improved travel safety and efficiency in complicated indoor environments.
Problem
Depth-sensor ETAs have limited range and cannot reliably measure transparent obstacles, while existing auditory feedback can leave blind users to determine the correct direction themselves.
Method
The system fuses depth and ultrasonic sensing to find traversable directions, delivers three auditory cues to totally blind users, and integrates direction information into AR visual enhancement for weak-sighted users.
Results
The fusion algorithm significantly improves accuracy for transparent obstacles, and the smart guiding glasses improve users' travel experience in complicated indoor environments.
Takeaways & Limitations
Smart guiding glasses can help visually impaired users move more safely and efficiently indoors through direct auditory or visual directional guidance.
Abstract
from arXiv · showhide
To overcome the travelling difficulty for the visually impaired group, this paper presents a novel ETA (Electronic Travel Aids)-smart guiding device in the shape of a pair of eyeglasses for giving these people guidance efficiently and safely. Different from existing works, a novel multi sensor fusion based obstacle avoiding algorithm is proposed, which utilizes both the depth sensor and ultrasonic sensor to solve the problems of detecting small obstacles, and transparent obstacles, e.g. the French door. For totally blind people, three kinds of auditory cues were developed to inform the direction where they can go ahead. Whereas for weak sighted people, visual enhancement which leverages the AR (Augment Reality) technique and integrates the traversable direction is adopted. The prototype consisting of a pair of display glasses and several low cost sensors is developed, and its efficiency and accuracy were tested by a number of users. The experimental results show that the smart guiding glasses can effectively improve the user's travelling experience in complicated indoor environment. Thus it serves as a consumer device for helping the visually impaired people to travel safely.
I. INTRODUCTION
The paper addresses indoor navigation challenges for visually impaired people by combining sensor-based obstacle avoidance with auditory or AR-based directional guidance. Its central contribution is a multi-sensor system designed to improve obstacle detection and guide users through traversable directions.
- Visually impaired people may not adequately perceive obstacle volume, distance, and surrounding information when relying on conventional white-cane navigation.
- Depth-sensor ETAs detect obstacles precisely but have limited working range and perform poorly with transparent objects such as glass doors and windows.
- Three auditory cues convert the computed traversable direction into direct guidance for totally blind users, reducing the need to interpret environmental feedback independently.
- For weak-sighted users, AR visual enhancement displays surrounding information and feasible directions on the glasses to support obstacle avoidance.
- The proposed obstacle-avoidance algorithm fuses depth and ultrasonic sensing to overcome transparent-obstacle detection limitations.
A. The Hardware System
The hardware system combines depth and ultrasonic sensing with onboard processing, AR glasses, and audio output. These components acquire environmental information, measure obstacle distance, compute guidance, and present it through visual or auditory interfaces.
- The prototype includes a depth camera, ultrasonic rangefinder, embedded CPU board, AR glasses, and an earphone.The CPU performs depth-image processing, sensor fusion, AR rendering, and guiding-sound synthesis.
- The depth sensor uses coded infrared illumination and a monochromatic CMOS image sensor to generate scene depth information.
- The ultrasonic rangefinder measures obstacle distance from the time interval between transmitted and reflected ultrasound.The sensor is mounted on the glasses and uses 40 KHz samples.
- The ultrasonic distance relation uses d = v · ToF / 2, where d is object distance, v is sound speed in air, and ToF is the transmit-receive interval.The passage gives v as 340 m/s.
B. The Steps
The system derives candidate traversable directions from depth imagery, selects a safe direction using ultrasonic fusion, and communicates it through AR or auditory cues.
- System workflow: The system processes depth images to generate candidate moving directions, then combines ultrasonic measurements to select an optimal direction.The overall pipeline also supports AR rendering and guiding-sound synthesis.
- Depth-based way-finding: The depth-based method samples a line near the bottom of the image to identify candidate traversable directions during gradual walking.An adaptive sliding window and traversable threshold are used to evaluate image regions.
- Depth-based way-finding: A region is treated as traversable when five continuous points satisfy the depth-noise range and exceed the distance threshold.Otherwise, the region is discarded as containing an obstacle.
- Direction selection: The algorithm stores steering angles for traversable regions and selects an optimal direction by balancing turning angle against traversable width.Smaller steering angles reduce turning effort, while wider regions are treated as safer.
- Guidance feedback: The fusion method addresses incorrect depth measurements caused when infrared sensing passes through transparent obstacles.The resulting direction is visualized through AR rendering or converted into stereo tones, recorded instructions, or frequency beeps.
- Multi-sensor fusion: Ultrasonic data are applied within the sensor’s 15° field of view to reject directions whose measured distance falls below the safety threshold.Directions outside that field, or beyond the threshold, remain available; otherwise the output is Null.
IV. EXPERIMENTAL RESULTS AND DISCUSSIONS
The system was evaluated with objective tests of adaptability, correctness, and computational cost, plus subjective tests of guiding efficiency across indoor scenarios.
- Evaluation design: The evaluation measured algorithm adaptability, correctness, and computational cost objectively, while subjective tests assessed guiding-instruction efficiency.These tests covered both system performance and user experience.
- Evaluation design: 20 users, including 10 people with amblyopia and 10 totally blind people, tested the system at home, office, and supermarket locations.Participants’ heights ranged from 1.5 m to 1.8 m.
A. Adaptability for Different Height
The adaptability experiment examined minimum detectable obstacle height across camera heights and distances to characterize performance for users of different heights.
- Adaptability for Different Height: More than 5 cm was the minimum obstacle height detectable by the proposed algorithm in the reported test.Obstacles were placed 1–2 m from cameras whose heights ranged from 1.4 m to 1.8 m.
- Adaptability for Different Height: Lower camera height enabled detection of smaller obstacles when obstacle distance was fixed.The experiment also varied obstacle distance to assess adaptability across conditions.
B. Correctness of Obstacle Avoiding Algorithm
Transparent-obstacle tests compared avoiding algorithms with and without ultrasonic sensing and found that ultrasonic fusion substantially improved detection accuracy for transparent glass.
- Correctness of Obstacle Avoiding Algorithm: 98.93% accuracy was achieved without ultrasonic sensing under frosted glass, but accuracy was very low under pure transparent glass.The failure under pure transparent glass is attributed to the depth camera’s sensing limitation.
- Correctness of Obstacle Avoiding Algorithm: The algorithm with ultrasonic sensing significantly improved accuracy and robustly detected and avoided obstacles across the transparent scenarios.The scenarios are illustrated in Fig. 13 and accuracy results in Fig. 14.
C. Computational Cost
The proposed system distributes computation across depth sensing, ultrasonic measurement and fusion, with AR rendering adding a small processing cost.
- 26.5 ms is the maximum time required for ultrasonic sensor measurement, depending on obstacle distance.
- 1.33 ms is the average time for multi-sensor fusion, and AR rendering takes 2.19 ms.
- The ultrasonic measurement runs on the MCU, while the system also performs depth processing, fusion and AR rendering.
- 8.23 ms is the average time for depth image acquisition, while way-finding takes 2.71 ms.
D. Interactive Experience
Interactive tests compared guiding cues and baseline mobility across home, office and supermarket scenarios. Performance depended on familiarity, environmental noise and the cue type used.
- Test scenarios: 40 m, 150 m and 1 km paths represented home, office and supermarket scenarios, respectively, with obstacles placed for testing.The home path included 10 obstacles ranging from 5 cm to 1 m; office and supermarket paths included 15 obstacles each.
- Totally blind users: In home and office settings, stereo tone and beep guidance required nearly the same time as using a cane.
- Totally blind users: Beep-based guidance was the most efficient and had the best adaptability across the tested scenarios.
- Totally blind users: In the unfamiliar supermarket, the proposed guidance was more efficient than a cane because it directly indicated where to go.
- Totally blind users: In noisy supermarkets, recorded left-right instructions outperformed stereo tones because users could not reliably identify direction from the tones.
- Weak-sighted users: Weak-sighted users were tested with AR guidance and without the glasses, using the same three scenarios.
FOR WEAK SIGHT USERS
For weak-sighted users, AR visual enhancement reduced collisions and improved travel performance particularly in unfamiliar indoor environments.
- The experiments verified that the smart guiding glasses were efficient, secure and helpful in complicated indoor environments.
V. CONCLUSION
The paper concludes that smart guiding glasses combine multi-sensor obstacle avoidance with auditory or AR guidance to support safer and more efficient indoor travel for visually impaired users.
- The depth image and multi-sensor fusion algorithms address avoidance of small and transparent obstacles.
- Beep-based auditory guidance was the most efficient and well-adapted option among the tested cues for totally blind users.
- AR visual enhancement integrates traversable direction into binocular images and helps weak-sighted users walk more quickly and safely.
- The computation is fast enough for obstacle detection and display, while the sensors are simple and low cost.
- Experimental results show improved travelling experience for visually impaired people in complicated indoor environments.