Blavigator: a cheap and reliable navigation aid for the blind

financed by

RIPD/ADA/109690/2009

 

Abstract
Navigation of blind people is extremely restricted because they use the stick for obstacle detection while following the front sides of houses and shops, memorising all locations they are becoming familiar with. In a new, unfamiliar setting they completely depend on people passing by to ask for a certain shop or the closest post office. Crossing a street is a challenge, after which they are again completely disoriented. In a society in which very sophisticated technology is available, from tracking GPS-RFID equipped containers in an area of hundreds of meters to GPS-GIS car navigation to Bluetooth emitting the sound of movie trailers to mobile phones in front of cinemas, one can question what it may cost to provide the blind with the most elementary technology to make life a little bit easier. This technology may not replace the stick, but should complement it: alert the user to obstacles a few meters away and provide guidance for going to a specific location down town or in a shopping mall. The goal of this project is to develop a prototype which costs less than 400 euro and which can be easily assembled by a technician. This system is lightweight, easy to put on and carry, i.e., not being a hindrance when walking with the stick, simple to use, and robust. The system consists of two small boxes and a wireless earphone-microphone unit. The first box, to be worn in front at breast height, contains two minicameras and an electronic compass. Its weight is negligeable and its width about 15cm. This box is connected to a second box with only one cable. The second box is worn in a side purse with a strap around the neck. It contains a powerful miniportable with WiFi and Bluetooth, a GPS receiver with antenna fixed to the strap, and an RFID sensor. This box measures about 4x20x20cm and weights about 1kg including batteries for an autonomy of at least 8 hours. An integral part of the prototype is a list of off-the-shelf components and step-by-step instructions for assemblage and software installation, including designs of the breast unit and side purse with strap such that they can be made using a preferred material, from denim to leather. All hardware being based on off-the-shelf components, like Bluetooth GPS, compass and RFID, also USB webcams, the main part of the project is devoted to software development, the functionality, and an extremely simple user interface. Concerning vision, there are two main functions: (1) To detect possible obstacles, in front at a distance of 2-3m, like loose or missing stones (Portuguese calcada), static objects like benches, poles, trash cans and plant pots, and moving objects like people and animals, especially those on collision course. (2) To aid in heading forward, indicating front sides of houses and shops, but also the centering on narrow sidewalks. To be discussed with end-users are additional functions, like detecting zebra road crossings which are not secured by traffic lights with audible signals, and staying within their borders while crossing. GPS, RFID and the compass serve to aid more global navigation using a GIS (geographic information system). The GIS resides on a remote server and queries are relayed by WiFi. However, GPS has a limited precision and may not be available in narrow streets and with clouded sky, and indoor reception is never possible. If WiFi reception is possible, a local GIS with a radius of about one hundred meters is updated on the system, such that at positions where WiFi is not available the user can still query the system. If both GPS and WiFi fail, the user can still be guided using the local GIS and the compass, but for calibrating the user's position it is necessary to mark important landmarks and shops by active RFID tags, using a sensor with a range of 2-3m. An implicit assumption of using WiFi and RFID technology is that authorities, both municipalities and managers of shopping malls, are willing to invest (little) money in it because of the benefits it brings to the blind and others with severe visual impairments. Most important is the design of the user interface. The system may not disturb normal navigation with the stick, it may not distract attention to normal sounds, and it must alert the user with a minimum of sounds. Likewise, the speech interface must be based on a minimum of queries and simple instructions. Right from the start of the project, ideas and solutions will be discussed with end users through ACAPO, the Portuguese association of the blind and amblyopes, and extensive field tests will be conducted. As for any new technology, blind people must learn how to use the system and become familiar with it, which is a learning process similar to that of using the stick. Only if blind people are willing to accept the system, this project will be a success. The participation of the blind through ACAPO is therefore an essential part of this project.

 

Participating Institutions
Principal Contractor and Research Unit

CINTAL logoCentro de Investigação Tecnológica do Algarve (CINTAL/UAlg)
Campus de Gambelas
8005-139Faro

 

Participating Institutions

CINTAL logo

Início

 

 

Universidade do Algarve (UALG)

Campus de Gambelas
8005-139 Faro

Universidade de Trás-os-Montes e Alto Douro

 

 

 

Universidade de Trás-os-Montes e Alto Douro (UTAD)
Quinta dos Prados - Apartado 1013
5000-911Vila Real
ACAPO - Associação dos Cegos e Amblíopes de Portugal

 

 

Associação dos Cegos e  Amblíopes de Portugal

(ACAPO)Av. D. Carlos I, n.º 126  9.º andar

Lisboa

1200-651Lisboa

Portugal

 

Publications

Updated 2012/08/30

Journal


Serrão, M., Shahrabadi, S., Moreno, M., José, J., Rodrigues, J.I., Rodrigues, J.M.F., du Buf, J.M.H. (2014) Computer vision and GIS for the navigation of blind persons in buildings, Int. J. Universal Access in the Information Society. doi:10.1007/s10209-013-0338-8 http://link.springer.com/article/10.1007/s10209-013-0338-8

Martins, J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2012) Local object gist: meaningful shapes and spatial layout at a very early stage of visual processing,Gestalt Theory, Vol. 34, No. 3/4, pp. 349-380 (ISSN 0170-057 X). http://w3.ualg.pt/~jrodrig/papers_pdf/Gestalt2012.pdf

Rodrigues, J.M.F., Lam, R., du Buf, J.M.H. (2012) Cortical 3D Face and Object Recognition Using 2D Projections, Int. J. of Creative Interfaces and Computer Graphics, Vol. 3, No 1, pp. 45-6. doi: 10.4018/jcicg.2012010104 http://w3.ualg.pt/~jrodrig/papers_pdf/ijcicg2012.pdf

Paulo Costa, Hugo Fernandes, Paulo Martins, João Barroso, Leontios J. Hadjileontiadis. (2014), “Obstacle detection and avoidance module for Blavigator system”, UAIS - Universal Access in the Information Society.

Costa, P.; Barroso J.; Fernandes, H.; Hadjileontiadis, L.; (2012). Using Peano-Hilbert space filling curves for fast bidimensional ensemble EMD realization, EURASIP Journal on Advances in Signal Processing.

Fernandes, H.; Conceição, N.; Paredes, H.; Pereira, A.; Araújo, P.; Barroso, J.; (2012). “Providing accessibility to blind people using GIS”, UAIS - Universal Access in the Information Society.

Conference/workshop 


Serrão, M., Rodrigues, J.M.F., du Buf, J.M.H. (2014) Navigation framework using visual landmarks and a GIS, Procedia Computer Science, vol. 27, pp.28–37. doi: 10.1016/j.procs.2014.02.005 http://www.sciencedirect.com/science/article/pii/S1877050914000076

(PT) Farrajota, M.,  Rodrigues, J.M.F., du Buf, J.M.H. (2013) Keypoint clustering using cortical colour opponency, In Proc. 19th Portuguese Conf. on Pattern Recognition, Lisboa, Portugal, 1 Nov., 2 pp., 2 pp. http://w3.ualg.pt/~jrodrig/papers_pdf/recpad2013b.pdf

Saleiro, M., Farrajota, M., Terzic, K., Rodrigues, João M.F., du Buf, J.M.H  (2013) A biological and realtime framework for hand gestures and head poses,In C. Stephanidis and M. Antona (Eds.) Universal Access in Human-Computer Interaction. Design Methods, Tools, and Interaction Techniques for eInclusion SE – 60, vol. 8009, pp. 556-565. Springer-Verlag Berlin Heidelberg. doi: 10.1007/978-3-642-39188-0_60. http://w3.ualg.pt/~jrodrig/papers_pdf/HCII2013.pdf

Shahrabadi, S., Rodrigues, J.M.F., Buf, J. M. H. (2013). Detection of Indoor and Outdoor Stairs. In J. Sanches, L. Micó, & J. Cardoso (Eds.), Pattern Recognition and Image Analysis SE - 100, vol. 7887, pp. 847–854. Springer Berlin Heidelberg. doi:10.1007/978-3-642-38628-2_100 http://w3.ualg.pt/~jrodrig/papers_pdf/ibpria2013a.pdf

Farrajota, M., Saleiro, S., Terzic, K., Rodrigues, J.M.H, du Buf, J.M.H  (2012) Multi-scale cortical keypoints for realtime hand tracking and gesture recognition, In Proc 1st Int. Workshop on Cognitive Assistive Systems: Closing the Action-Perception Loop (ISBN 978-972-8822-26-2) in conjunction with  IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Vilamoura, Portugal, 7-12 Oct., pp. 9-15. http://w3.ualg.pt/~jrodrig/papers_pdf/SW5_2012.pdf

Martins, J.A. Farrajota, M., Lam, R., Rodrigues, J.M.F, Terzic, K., du Buf, J.M.H. (2012) A disparity energy model improved by line, edge and keypoint correspondences, In Proc European Conf. on Visual Perception, Alghero, Italy, 2-6 Sept., Perception Vol. 41 Suppl., pp. 76 http://w3.ualg.pt/~jrodrig/papers_pdf/ecvp2012a.pdf

José, J., Moreno, M. Pinilla-Dutoit, J., Rodrigues, J.M.F, du Buf, J.M.H. (2012) Blavigator: a navigation aid for blind persons, In Proc. European Conf. on Visual Perception, Alghero, Italy, 2-6 Sept., Perception Vol. 41 Suppl., pp. 99. http://w3.ualg.pt/~jrodrig/papers_pdf/ecvp2012c.pdf

Moreno, M., Shahrabadi, S., José, J., du Buf, J.M.H. and Rodrigues, J.M.F. (2012) Realtime local navigation for the blind: detection of lateral doors and sound interface, Procedia Computer Science Vol. 14, pp. 74 – 82. doi: 10.1016/j.procs.2012.10.009 http://w3.ualg.pt/~jrodrig/papers_pdf/dsai2012m.pdf

Serrão, M., Rodrigues, J.M.F., Rodrigues, J.I. and du Buf, J.M.H (2012) Indoor localization and navigation for blind persons using visual landmarks and a GIS, Procedia Computer Science, Vol. 14, pp. 65 – 73. doi: 10.1016/j.procs.2012.10.008 http://w3.ualg.pt/~jrodrig/papers_pdf/dsai2012s.pdf

Rodrigues, J. M. F., Martins, J. A., Lam, R., Buf, J. M. H. (2012). Cortical Multiscale Line-Edge Disparity Model. In A. Campilho & M. Kamel (Eds.), Image Analysis and Recognition SE  - 35, vol. 7324, pp. 296–303. Springer Berlin Heidelberg. doi:10.1007/978-3-642-31295-3_35 http://w3.ualg.pt/~jrodrig/papers_pdf/iciar2012.pdf

Saleiro, M., Rodrigues, J.M.F. and du Buf, J.M.H. (2012) Minimalistic Vision-based Cognitive SLAM, In Proc. 4th Int. Conf. on Agents and Artificial Intelligence, Special Session Intelligent Robotics (ICAART-SSIR2012), Vilamoura, Portugal, 6-8 Fev., Vol. 1, pp. 614-623. http://w3.ualg.pt/~jrodrig/papers_pdf/icaart2012.pdf

José, J., Rodrigues, J.M.F., du Buf, J.M.H. (2012) Visual navigation for the blind: path and obstacle detection, In Proc. Int. Conf. on Pattern Recognition Applications and Methods (ICPRAM 2012), Vilamoura, Portugal, 6-8 Fev. Vol. 2, pp. 515-519. http://w3.ualg.pt/~jrodrig/papers_pdf/icpram2012.pdf

Martins, J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2011) Disparity energy model using a trained neuronal population, In Proc. IEEE Int. Symp. on Signal Proc. and Info. Technology (ISSPIT 2011), Bilbao, Spain, 14-17 Dec. pp. 287-292. doi: 10.1109/ISSPIT.2011.6151575 http://w3.ualg.pt/~jrodrig/papers_pdf/isspit2011.pdf

[PT] Farrajota, M., Martins, J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2011) Disparity energy model with keypoint disparity validation, In Proc. 17th Portuguese Conf. on Pattern Recognition, Porto, Portugal, 28 Oct.. pp. 70-71 http://w3.ualg.pt/~jrodrig/papers_pdf/recpad2011.pdf

Hugo Fernandes; Vitor Filipe; Paulo Costa; João Barroso, (2013), “Location based services for the blind supported by RFID technology”, 5th International Conference on DSAI2013 - Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, Vigo, Spain, 2013. No prelo de Procedia Computer Science Journal, Elsevier. (Submetido no período do relatório) (publicado no volume 27, início de 2013). http://www.sciencedirect.com/science/article/pii/S1877050914000040

Hugo Fernandes, Jose Faria, Paulo Martins, Hugo Paredes, João Barroso, (2013). “RFID Mesh Network as an Infrastructure for Location Based Services for the Blind”, HCI International 2013, Las Vegas, NV, USA, 2013. ISBN: 978-3-642-39341-9. No prelo de 15th International Conference, HCI International 2013, Springer. http://link.springer.com/chapter/10.1007%2F978-3-642-39342-6_5

Paredes, Hugo; Fernandes, Hugo; Martins, Paulo; Barroso, João. 2013. Gathering the Users’ Needs in the Development of Assistive Technology: A Blind Navigation System Use Case.  In Universal Access in Human-Computer Interaction. Applications and Services for Quality of Life, ed. Constantine Stephanidis, Margherita Antona, 79 - 88. ISBN: 978-3-642-39193-4. Berlin, Heidelberg: Springer Berlin Heidelberg. doi: 10.1007/978-3-642-39194-1_10. (http://link.springer.com/chapter/10.1007%2F978-3-642-39342-6_5 ).

Paulo Costa; Hugo Fernandes; Paulo Martins; João Barroso; Leontios Hadjileontiadis; (2012). “Obstacle detection using stereo imaging to assist the navigation of visually impaired people”. DSAI 2012 – 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion, Porto, Portugal, 2012. ISBN: 978-989-704-089-4. No prelo de Procedia Computer Science Journal, Elsevier.  http://www.sciencedirect.com/science/article/pii/S1877050912007727

Vítor Filipe, Filipe Fernandes, Hugo Fernandes, António Sousa, Hugo Paredes, João Barroso; (2012). “Blind navigation support system based on Microsoft Kinect”. DSAI 2012 – 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion, Porto, Portugal, 2012. ISBN: 978-989-704-089-4. No prelo de Procedia Computer Science Journal, Elsevier.  http://www.sciencedirect.com/science/article/pii/S1877050912007739

Fernandes, H.; Adão, T.; Magalhães, L.; Paredes, H.; Barroso, J.. 2012. "Navigation Module of Blavigator Prototype", World Automation Congress 2012, In Proceedings of the World Automation Congress, Puerto Vallarta. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=6320942

Fernandes, H.; Costa, P.; Vasconcelos, V.; Coelho, P.; Barroso, J.; Hadjileontiadis, L.. 2011. "Landmarks detection to assist the navigation of visually impaired people", HCI International 2011, In HCI International 2011 Conference Proceedings, Orlando, Florida, US. http://link.springer.com/chapter/10.1007%2F978-3-642-21616-9_33

Fernandes, H.; Henriques, F.; Paredes, H.; Martins, P.; Barroso, J.. 2011. "A prototype for a blind navigation system", World Conference on Information Technology, Antalya. Turkey. http://goo.gl/TLxx29

Fernandes, Hugo; Faria, José; Paredes, Hugo; Barroso, João. 2011. "An integrated system for blind day-to-day life autonomy", The proceedings of the 13th international ACM SIGACCESS conference, In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility - ASSETS '11, Dundee, Scotland, UK. http://dl.acm.org/citation.cfm?id=2049579

 

Copyright Notice


The publications presented here are provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Single copies of these articles can be downloaded and printed only for the reader's personal research and study. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Some Results

The previous system

Different pavements and detected objects UALG - Gambelas Campus

Detection of moving obstacles (like persons)

Cane with RFID reader in UTAD campus