As retina specialists know, conceptualizing and navigating a physical environment poses a real challenge to blind individuals, who are unable to capture sensory information in the way sighted individuals do. In a recent study published in the Journal of Visualized Experiments, researchers at the Massachusetts Eye and Ear Infirmary (MEEI), Harvard Medical School, and collaborators at the University of Chile explored the use of a virtual game to help blind patients improve their navigation skills and develop a cognitive spatial map of a local building. In an interview with Retina Today, Lotfi B. Merabet, OD, PhD, MPH, the study's senior author, described the concept behind the investigators' virtual gaming environment and the results observed so far in their study.

AbES Software

“As sighted individuals, we can create a mental map of the world around us for the purposes of finding our way around,” Dr. Merabet, Director of the Laboratory for Visual Neuroplasticity at MEEI, told Retina Today. “Vision is very powerful for capturing relevant spatial information such as important landmarks. The question becomes, how do blind people do this in the absence of sight? Imagine further if you were born blind and have never seen.”

As Dr. Merabet explained, evidence suggests that blind individuals can create spatial mental maps using other sensory channels such as hearing and touch. Recognizing the potential of this ability, the researchers developed software called the Audio-based Environmental Simulator (AbES), designed to improve real-world navigation skills in the blind.

“Basically, the approach attempts to leverage the principal of learning by stimulation, eg, using a flight simulator to learn how to fly,” Dr. Merabet explained. “A blind individual learns the spatial layout of a building (which they have never visited) by exploring the environment virtually. The person builds a mental map of that space that they can use for the purposes of navigating the real environment represented in the virtual simulation.”

The virtual game developed by Dr. Merabet and colleagues at the University of Chile is based on a building at The Carroll Center for the Blind in Newton, MA. In the game, participants must find jewels and carry them out of the building without being intercepted by roving monsters. Players utilize a keyboard and headphones that play auditory clues to help them generate an accurate mental layout of the virtual building.

Preliminary Study Results

Dr. Merabet and colleagues are currently evaluating the AbES software in an ongoing study of blind men and women aged 18 to 45 years. All participants are legally blind of either early onset (documented prior to age 3 years) or late onset (occurring after age 14 years) from varying ocular etiologies. None of the participants were previously familiar with the layout of the physical building represented in the virtual game. After playing the game, participants are assessed on their ability to navigate within the physical building.

The study reported preliminary results for 3 study participants aged 19 to 22 years. The investigators found that all 3 participants showed a high level of success on all 3 navigation tasks following game play with the AbES software. This success was confirmed by the performance scores (group and individual) on 3 behavioral tasks, the study authors said. The percentage correct performance for the virtual (mean, 90%) followed by the physical (mean, 88.7%) navigation tasks illustrated a high level of success and comparable performance for both tasks, the investigators concluded.

After completing the game, participants were able to find their way around the building in real life. They were also better at finding alternative paths in the building than other blind people who had been taught the layout in a step-by-step fashion.

“We have found that even when individuals interact with the environment within the context of a video game metaphor (searching for hidden jewels in rooms), they can also learn the spatial layout implicitly just as well as someone who was taught the step-by-step layout,” Dr. Merabet said. “Furthermore, we have also found that learning the layout by gaming creates a more robust mental representation of the spatial layout. That is, patients are able to determine alternate routes that are not learned by explicit path learning. That is the gaming advantage.”

Dr. Merabet noted that, interestingly, patients often report that once they arrive at the physical building, they have the impression that it is smaller than they previously imagined.

Future Work

The researchers are reportedly exploring applications of this technology to incorporate tactile forms of user interfaces such as the Wiimote (Nintendo) or joystick.

“We are scaling up the approach to incorporate a large-scale indoor and outdoor representation of The Carroll Center for the Blind. In this way, clients and students have a means to explore the campus facilities before they arrive at the center as a way to familiarize themselves with the overall layout,” Dr. Merabet said.

  1. Connors EC, Yazzolino LA, Sanchez J, Merabet LB. Development of an audio-based virtual gaming environment to assist with navigation skills in the blind. J Vis Exp. 2013;(73):e50272.