Blavigator: a cheap and reliable navigation aid for the blind
financed by
RIPD/ADA/109690/2009
| Abstract |
|
Navigation of blind people
is extremely restricted because they use the stick for obstacle
detection while following the front sides of houses and shops,
memorising all locations they are becoming familiar with. In a new,
unfamiliar setting they completely depend on people passing by to ask
for a certain shop or the closest post office. Crossing a street is a
challenge, after which they are again completely disoriented. In a
society in which very sophisticated technology is available, from
tracking GPS-RFID equipped containers in an area of hundreds of meters
to GPS-GIS car navigation to Bluetooth emitting the sound of movie
trailers to mobile phones in front of cinemas, one can question what it
may cost to provide the blind with the most elementary technology to
make life a little bit easier. This technology may not replace the stick,
but should complement it: alert the user to obstacles a few meters away
and provide guidance for going to a specific location down town or in a
shopping mall. The goal of this project is to develop a prototype which
costs less than 400 euro and which can be easily assembled by a
technician. This system is lightweight, easy to put on and carry, i.e.,
not being a hindrance when walking with the stick, simple to use, and
robust. The system consists of two small boxes and a wireless
earphone-microphone unit. The first box, to be worn in front at breast
height, contains two minicameras and an electronic compass. Its weight
is negligeable and its width about 15cm. This box is connected to a
second box with only one cable. The second box is worn in a side purse
with a strap around the neck. It contains a powerful miniportable with
WiFi and Bluetooth, a GPS receiver with antenna fixed to the strap, and
an RFID sensor. This box measures about 4x20x20cm and weights about 1kg
including batteries for an autonomy of at least 8 hours. An integral
part of the prototype is a list of off-the-shelf components and
step-by-step instructions for assemblage and software installation,
including designs of the breast unit and side purse with strap such that
they can be made using a preferred material, from denim to leather. All
hardware being based on off-the-shelf components, like Bluetooth GPS,
compass and RFID, also USB webcams, the main part of the project is
devoted to software development, the functionality, and an extremely
simple user interface. Concerning vision, there are two main functions:
(1) To detect possible obstacles, in front at a distance of 2-3m, like
loose or missing stones (Portuguese calcada), static objects like
benches, poles, trash cans and plant pots, and moving objects like
people and animals, especially those on collision course. (2) To aid in
heading forward, indicating front sides of houses and shops, but also
the centering on narrow sidewalks. To be discussed with end-users are
additional functions, like detecting zebra road crossings which are not
secured by traffic lights with audible signals, and staying within their
borders while crossing. GPS, RFID and the compass serve to aid more
global navigation using a GIS (geographic information system). The GIS
resides on a remote server and queries are relayed by WiFi. However, GPS
has a limited precision and may not be available in narrow streets and
with clouded sky, and indoor reception is never possible. If WiFi
reception is possible, a local GIS with a radius of about one hundred
meters is updated on the system, such that at positions where WiFi is
not available the user can still query the system. If both GPS and WiFi
fail, the user can still be guided using the local GIS and the compass,
but for calibrating the user's position it is necessary to mark
important landmarks and shops by active RFID tags, using a sensor with a
range of 2-3m. An implicit assumption of using WiFi and RFID technology
is that authorities, both municipalities and managers of shopping malls,
are willing to invest (little) money in it because of the benefits it
brings to the blind and others with severe visual impairments. Most
important is the design of the user interface. The system may not
disturb normal navigation with the stick, it may not distract attention
to normal sounds, and it must alert the user with a minimum of sounds.
Likewise, the speech interface must be based on a minimum of queries and
simple instructions. Right from the start of the project, ideas and
solutions will be discussed with end users through ACAPO, the Portuguese
association of the blind and amblyopes, and extensive field tests will
be conducted. As for any new technology, blind people must learn how to
use the system and become familiar with it, which is a learning process
similar to that of using the stick. Only if blind people are willing to
accept the system, this project will be a success. The participation of
the blind through ACAPO is therefore an essential part of this project.
|
| Participating Institutions |
|
Principal Contractor and Research
Unit
Participating Institutions
Universidade do Algarve (UALG)
Campus de Gambelas
Universidade
de Trás-os-Montes e Alto Douro (UTAD)
Associação dos Cegos e Amblíopes de Portugal (ACAPO)Av. D. Carlos I, n.º 126 9.º andar Lisboa 1200-651Lisboa Portugal
|
| Publications |
|
Updated 2012/08/30 Journal
Serrão, M., Shahrabadi, S., Moreno, M., José, J.,
Rodrigues, J.I., Rodrigues,
J.M.F.,
du Buf, J.M.H. (2014) Computer
vision and GIS for the navigation of blind persons in buildings,
Int. J. Universal Access in the Information Society. doi:10.1007/s10209-013-0338-8
http://link.springer.com/article/10.1007/s10209-013-0338-8
Martins,
J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2012) Local
object gist: meaningful shapes and spatial layout at a very early stage
of visual processing,Gestalt
Theory, Vol. 34, No. 3/4, pp. 349-380 (ISSN 0170-057 X).
http://w3.ualg.pt/~jrodrig/papers_pdf/Gestalt2012.pdf Rodrigues,
J.M.F., Lam, R., du Buf, J.M.H. (2012) Cortical
3D Face and Object Recognition Using 2D Projections, Int. J. of
Creative Interfaces and Computer Graphics, Vol. 3, No 1, pp. 45-6. doi:
10.4018/jcicg.2012010104
http://w3.ualg.pt/~jrodrig/papers_pdf/ijcicg2012.pdf Paulo Costa, Hugo Fernandes, Paulo Martins, João Barroso, Leontios J. Hadjileontiadis. (2014), “Obstacle detection and avoidance module for Blavigator system”, UAIS - Universal Access in the Information Society. Costa, P.; Barroso J.; Fernandes, H.; Hadjileontiadis, L.; (2012). Using Peano-Hilbert space filling curves for fast bidimensional ensemble EMD realization, EURASIP Journal on Advances in Signal Processing. Fernandes, H.; Conceição, N.; Paredes, H.; Pereira, A.; Araújo, P.; Barroso, J.; (2012). “Providing accessibility to blind people using GIS”, UAIS - Universal Access in the Information Society. Conference/workshop
Serrão, M., Rodrigues,
J.M.F., du Buf, J.M.H. (2014) Navigation framework using visual landmarks and a GIS,
Procedia Computer Science, vol. 27, pp.28–37.
doi: 10.1016/j.procs.2014.02.005 http://www.sciencedirect.com/science/article/pii/S1877050914000076
(PT) Farrajota, M., Rodrigues,
J.M.F., du Buf, J.M.H. (2013) Keypoint clustering using cortical colour opponency, In
Proc. 19th
Portuguese Conf. on Pattern Recognition, Lisboa, Portugal, 1 Nov., 2
pp., 2 pp.
http://w3.ualg.pt/~jrodrig/papers_pdf/recpad2013b.pdf
Saleiro, M., Farrajota, M., Terzic, K., Rodrigues, João M.F.,
du Buf, J.M.H (2013) A
biological and realtime framework for hand gestures and head poses,In
C. Stephanidis and M. Antona (Eds.) Universal Access in Human-Computer
Interaction. Design Methods, Tools, and Interaction Techniques for
eInclusion SE – 60, vol. 8009, pp. 556-565. Springer-Verlag Berlin
Heidelberg. doi: 10.1007/978-3-642-39188-0_60.
http://w3.ualg.pt/~jrodrig/papers_pdf/HCII2013.pdf
Shahrabadi, S., Rodrigues, J.M.F., Buf,
J. M. H. (2013). Detection
of Indoor and Outdoor Stairs.
In J. Sanches, L. Micó, & J. Cardoso (Eds.), Pattern Recognition and
Image Analysis SE - 100, vol. 7887, pp. 847–854. Springer Berlin
Heidelberg. doi:10.1007/978-3-642-38628-2_100
http://w3.ualg.pt/~jrodrig/papers_pdf/ibpria2013a.pdf
Farrajota,
M., Saleiro, S., Terzic, K., Rodrigues, J.M.H, du Buf, J.M.H
(2012) Multi-scale
cortical keypoints for realtime hand tracking and gesture recognition, In
Proc 1st Int. Workshop on Cognitive Assistive Systems: Closing the
Action-Perception Loop (ISBN 978-972-8822-26-2) in conjunction with
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems,
Vilamoura, Portugal, 7-12 Oct., pp. 9-15.
http://w3.ualg.pt/~jrodrig/papers_pdf/SW5_2012.pdf Martins,
J.A. Farrajota, M., Lam, R., Rodrigues, J.M.F, Terzic, K., du Buf,
J.M.H. (2012) A
disparity energy model improved by line, edge and keypoint
correspondences, In Proc European Conf. on Visual Perception,
Alghero, Italy, 2-6 Sept., Perception Vol. 41 Suppl., pp. 76
http://w3.ualg.pt/~jrodrig/papers_pdf/ecvp2012a.pdf José, J.,
Moreno, M. Pinilla-Dutoit, J., Rodrigues, J.M.F, du Buf, J.M.H. (2012) Blavigator:
a navigation aid for blind persons, In Proc. European Conf. on
Visual Perception, Alghero, Italy, 2-6 Sept., Perception Vol. 41 Suppl.,
pp. 99.
http://w3.ualg.pt/~jrodrig/papers_pdf/ecvp2012c.pdf
Moreno, M., Shahrabadi,
S., José, J., du Buf, J.M.H. and Rodrigues, J.M.F. (2012) Realtime
local navigation for the blind: detection of lateral doors and sound
interface, Procedia
Computer Science Vol. 14, pp. 74 – 82. doi: 10.1016/j.procs.2012.10.009
http://w3.ualg.pt/~jrodrig/papers_pdf/dsai2012m.pdf Serrão, M.,
Rodrigues, J.M.F., Rodrigues, J.I. and du Buf, J.M.H (2012) Indoor
localization and navigation for blind persons using visual landmarks and
a GIS, Procedia
Computer Science, Vol. 14, pp. 65 – 73. doi: 10.1016/j.procs.2012.10.008
http://w3.ualg.pt/~jrodrig/papers_pdf/dsai2012s.pdf
Rodrigues, J. M. F., Martins, J. A., Lam, R., Buf, J. M. H. (2012). Cortical Multiscale Line-Edge Disparity Model. In A. Campilho & M. Kamel (Eds.), Image
Analysis and Recognition SE - 35, vol. 7324, pp. 296–303. Springer
Berlin Heidelberg. doi:10.1007/978-3-642-31295-3_35
http://w3.ualg.pt/~jrodrig/papers_pdf/iciar2012.pdf Saleiro, M.,
Rodrigues, J.M.F. and du Buf, J.M.H. (2012) Minimalistic
Vision-based Cognitive SLAM, In Proc. 4th Int. Conf. on Agents
and Artificial Intelligence, Special Session Intelligent Robotics
(ICAART-SSIR2012), Vilamoura, Portugal, 6-8 Fev., Vol. 1, pp. 614-623.
http://w3.ualg.pt/~jrodrig/papers_pdf/icaart2012.pdf José, J.,
Rodrigues, J.M.F., du Buf, J.M.H. (2012) Visual
navigation for the blind: path and obstacle detection, In Proc.
Int. Conf. on Pattern Recognition Applications and Methods (ICPRAM
2012), Vilamoura, Portugal, 6-8 Fev. Vol. 2, pp. 515-519.
http://w3.ualg.pt/~jrodrig/papers_pdf/icpram2012.pdf
Martins,
J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2011) Disparity
energy model using a trained neuronal population, In Proc. IEEE
Int. Symp. on Signal Proc. and Info. Technology (ISSPIT 2011), Bilbao,
Spain, 14-17 Dec. pp.
287-292. doi:
10.1109/ISSPIT.2011.6151575
http://w3.ualg.pt/~jrodrig/papers_pdf/isspit2011.pdf
[PT]
Farrajota, M., Martins,
J.A., Rodrigues, J.M.F. and du Buf, J.M.H. (2011) Disparity
energy model with keypoint disparity validation, In Proc. 17th
Portuguese Conf. on Pattern Recognition, Porto, Portugal, 28 Oct.. pp.
70-71
http://w3.ualg.pt/~jrodrig/papers_pdf/recpad2011.pdf
Hugo Fernandes; Vitor Filipe; Paulo Costa; João Barroso, (2013),
“Location based services for the blind supported by RFID technology”,
5th International Conference on DSAI2013 - Software Development and
Technologies for Enhancing Accessibility and Fighting Info-exclusion,
Vigo, Spain, 2013. No prelo de
Procedia Computer Science Journal, Elsevier. (Submetido no período do
relatório) (publicado no volume 27, início de 2013).
http://www.sciencedirect.com/science/article/pii/S1877050914000040 Hugo Fernandes, Jose Faria, Paulo Martins, Hugo Paredes, João Barroso,
(2013). “RFID
Mesh Network as an Infrastructure for Location Based Services for the
Blind”, HCI International 2013, Las Vegas, NV, USA, 2013.
ISBN: 978-3-642-39341-9. No prelo de 15th
International Conference, HCI International 2013, Springer.
http://link.springer.com/chapter/10.1007%2F978-3-642-39342-6_5 Paredes, Hugo;
Fernandes, Hugo; Martins, Paulo; Barroso, João. 2013.
Gathering the Users’ Needs in the Development of
Assistive Technology: A Blind Navigation System Use Case.
In Universal Access in Human-Computer Interaction. Applications
and Services for Quality of Life, ed. Constantine Stephanidis,
Margherita Antona, 79 - 88. ISBN: 978-3-642-39193-4. Berlin, Heidelberg:
Springer Berlin Heidelberg. doi: 10.1007/978-3-642-39194-1_10.
(http://link.springer.com/chapter/10.1007%2F978-3-642-39342-6_5
). Paulo Costa;
Hugo Fernandes; Paulo Martins; João Barroso; Leontios Hadjileontiadis;
(2012).
“Obstacle detection using stereo imaging to assist the navigation of
visually impaired people”. DSAI 2012 – 4th International Conference on
Software Development for Enhancing Accessibility and Fighting
Info-exclusion, Porto, Portugal, 2012.
ISBN: 978-989-704-089-4. No prelo de
Procedia Computer Science Journal, Elsevier.
http://www.sciencedirect.com/science/article/pii/S1877050912007727 Vítor Filipe, Filipe Fernandes, Hugo Fernandes, António Sousa, Hugo Paredes,
João Barroso; (2012).
“Blind navigation support system based on Microsoft Kinect”. DSAI 2012 –
4th International Conference on Software Development for Enhancing
Accessibility and Fighting Info-exclusion, Porto, Portugal, 2012.
ISBN: 978-989-704-089-4. No prelo de
Procedia Computer Science Journal, Elsevier.
http://www.sciencedirect.com/science/article/pii/S1877050912007739 Fernandes, H.;
Adão, T.; Magalhães, L.; Paredes, H.; Barroso, J..
2012. "Navigation
Module of Blavigator Prototype", World Automation Congress 2012, In
Proceedings of the World Automation Congress, Puerto Vallarta.
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=6320942 Fernandes, H.;
Costa, P.; Vasconcelos, V.; Coelho, P.; Barroso, J.; Hadjileontiadis,
L.. 2011.
"Landmarks detection to assist the navigation of visually impaired
people", HCI International 2011, In HCI International 2011 Conference
Proceedings, Orlando, Florida, US.
http://link.springer.com/chapter/10.1007%2F978-3-642-21616-9_33 Fernandes, H.;
Henriques, F.; Paredes, H.; Martins, P.; Barroso, J..
2011. "A prototype
for a blind navigation system", World Conference on Information
Technology, Antalya. Turkey.
http://goo.gl/TLxx29 Fernandes, Hugo;
Faria, José; Paredes, Hugo; Barroso, João. 2011.
"An integrated system
for blind day-to-day life autonomy", The proceedings of the 13th
international ACM SIGACCESS conference, In The proceedings of the 13th
international ACM SIGACCESS conference on Computers and accessibility -
ASSETS '11, Dundee, Scotland, UK.
http://dl.acm.org/citation.cfm?id=2049579
Copyright Notice The publications presented here are provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Single copies of these articles can be downloaded and printed only for the reader's personal research and study. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder. |
| Some Results |
|
The previous system
Different pavements and detected objects UALG - Gambelas Campus
Detection of moving obstacles (like persons)
Cane with RFID reader in UTAD campus
|
