SmartVision: active vision for the blind

financed by

FCT/MCTES - PTDC/EIA/73633/2006



Participating Institutions

Project Objectives


Public data


Abstract    Scheduling

Every car and bicycle can be equipped with a GPS/GIS-based navigation system that may cost a few hundreds of euros. By contrast, blind and visually impaired persons need to navigate using the stick or, at best, an ultrasonic obstacle detector. This asymmetry needs to be solved, because there are an estimated 180 million persons with severe impairments of which 40-50 million are completely blind, and every year 2 million people more become blind.

The Greek project SmartEyes aims at developing a portable GPS/GIS-based navigation aid for the blind, obviously limited to outdoor navigation and stable landmarks like zebra crossings, bus stops, shops etc. This FCT project concerns a collaboration with the SmartEyes project, for developing an active vision module. We will develop the necessary technology for outdoor and indoor navigation, with obstacle avoidance and object recognition. Main problems to be solved are the detection of outlines of sidewalks and zebra crossings while walking, including nearby obstacles like trees, traffic signs, persons (children) and dogs. Indoor navigation requires detection of corridors, doors and furniture, but also locating objects, like a ketchup bottle in a pantry or fridge. Only by complementing SmartEyes with realtime vision it will become a really smart aid!

The necessary hardware is already available: a palmtop with BlueTooth link and GPS (in the future Galileo), plus cameras with USB interface. The main problem of a palmtop is its processor without floating-point units, which means that state-of-the-art algorithms from computer vision must be implemented using integers. There are two solutions: (a) most of the processing can be done remotely on a normal PC or even a small cluster, and (b) instead of applying most processing from the first to the last pixel of the video frames, processing can be limited to important regions by using a model of human vision, namely Focus-of-Attention (FoA).

Instead of only focusing on computer vision, this project establishes a link with human vision. Recent progress in the modelling of processes in the visual cortex showed that computational models can already compete with the state-of-the-art in computer vision. This concerns 2D multi-scale line/edge and keypoint extraction, Focus-of-Attention and object categorisation.

Taking into account the increasing performance of computers, more advanced and complete models can provide solutions to the real-world problems that this project addresses. One of the many innovative aspects of this project is therefore to extend cortical models with depth processing, i.e. disparity (stereo) and shape-from-shading and from texture gradients.

Linking computer vision and human vision allows to "combine the best of two worlds". For example, recognition of rotated 3D objects by using a limited number of views in visual memory is a difficult problem to be solved because of outliers, but robust methods are available from computer vision. As a result, the risc of not achieving good results is low. In addition to bringing together researchers in computer and human vision, a special collaboration concerns visual psychophysics: (1) measuring fixation points by eye-tracking allows to optimise the FoA model, such that 3D feature extraction and object segregation and matching can be improved and accelerated; (2) fMRI/EEG studies can reveal which cortical areas are involved when processing objects with and without different cues (colour, shading), such that coarse-to-fine-scale processing models can be optimised.

This project brings together researchers who can combine their expertise in building portable prototype systems with speech/audio interfaces, plus the state of the art in computer and human vision, including (psycho)physicists specialised in eye-tracking and fMRI/EEG experiments. The fact that state-of-the-art knowledge is already available, and active collaborations with other leading research groups are planned, the state of the art in the different specialisations will be significantly advanced. In addition, towards the end of the project the performance of the prototype system will be showcased to a wider audience, including TV and organisations of the blind, and a developer of professional systems will be attracted for production and marketing. With an estimated 180 million severely impaired persons, there is definitely a serious market for an intelligent navigation aid!

  • 1st Meeting

    4 January 2008

    IBILI, Coimbra


  • 2nd Meeting

    9 June 2009

    UALG, Faro


  • 3rd Meeting

    December 2009

    UTAD, Vila Real




HomeVision Laboratory

Updated: 01-10-2009