Seeing with your tongue
By RON SEELY
608-252-6131
Roger Behm lost his sight at 16, the victim of an inherited disease that destroyed
his retinas. Both of his eyes were surgically removed.
Now 55, Behm has made himself at home in a sightless world. He started his own business
in Janesville selling devices that help the blind cope with day-to-day tasks. He
and his wife have raised five children and just adopted another child from China
who is also blind. He fishes, canoes, camps and scuba dives.
But Behm can remember seeing. Which is why he couldn’t believe it when, three years
ago, he slipped a device over his head, turned it on, and was once again able to
discern light and dark, shapes and shadows, letters and numbers, and even a rolling
golf ball.
Links
• To learn more about BrainPort and Wicab, Inc. go to the company’s site
• Erik Weihenmayer's page
• For more on sensory substitution, see this article from The Why Files
• GRAPHIC: How the technology works
"I could look down and and see the ball, white on black, and I could see myself swinging
my putter," Behm said. "And, of course, I missed. But I could reach down and pick
up my ball, like any other sighted person."
The device is called BrainPort and, though it seems like a gadget from Star Trek,
it may be available commercially by the end of the year.
It works by converting images from a video camera to electrical impulses that are
transmitted via the tongue to the brain of the blind person and turned again into
black-and-white images that the user sees.
It takes advantage of groundbreaking work by a UW-Madison scientist that showed the
brain will reprogram itself to accept and use different sensory signals — in this
case touch instead of sight — to replace signals that can no longer be received due
to injury or disease.
The device, which consists of a miniature camera mounted on a pair of sunglasses,
a tongue sensor and a small control unit, was developed by Wicab of Middleton. It
builds on another of the company’s devices that uses the same underlying ideas to
help restore users’ balance.
The company is applying to the federal Food and Drug Administration to get approval
for a marketable version of the vision device that could be available by the end
of the year, Wicab CEO Robert Beckman said.
Trying circumstances
Few have tested BrainPort under more trying circumstances than Erik Weihenmayer,
the only blind man to reach the summit of Mt. Everest. Weihenmayer, totally blind
since the age of 16, has used the device to help him hike in the woods, even ascend
climbing walls. But he has most appreciated it for letting him do such simple but
rewarding tasks as playing tic-tac-toe with his daughter or reaching down to pet
his dog.
"I have a climbing friend who didn’t believe me when I told him about this," Weihenmayer
said. "So he put a Pepsi can on my table in my kitchen while I was out of the room.
Then he called me back in and told me to grab it. I reached out and grabbed the Pepsi
can. He was blown away. He was speechless. He had tears in his eyes.
"I mean, it may not seem like a real big deal to people, but to be able to see your
coffee cup ... ."
Neither Behm nor Weihenmayer are paid consultants to Wicab, although the company
pays some of their expenses.
The late Paul Bach-y-Rita, a UW-Madison physician and specialist in rehabilitation,
first came up with the ideas that inspired BrainPort in the 1960s. The technology
was patented by UW-Madison in 1998, and commercial development has been under way
for more than 10 years.
New ways to work
Bach-y-Rita’s earliest thinking about the brain’s ability to adapt to new ways of
receiving and processing information — its "plasticity," as it is known now — was
likely sparked by the dramatic struggle of his father, Pedro, to recover from a devastating
stroke in the mid-1960s, Beckman said.
Neurologists in those days believed brain damage could not be reversed. But Bach-y-Rita’s
brother, George, soon put their father to work doing chores such as sweeping the
porch of the house. Forced to accomplish more and more difficult tasks, their father
eventually recovered completely and even went back to his job teaching.
He died at the age of 73 of a heart attack while climbing in the mountains of Columbia.
Remarkably, studies of Pedro’s brain after his death showed massive damage to his
brain from the stroke. Yet he recovered. Somehow, his brain had found new ways to
work.
At the UW-Madison, Bach-y-Rita focused his studies on sensory substitution, the idea
that the brain can learn how to use other senses to replace one that has been lost
or damaged. He concentrated on the power of touch, studying what happens in the brain
when visual cues come from the sensitive nerves of the skin, such as those on the
fingertips.
Perfect organ
Those studies buttressed others that showed the brain can indeed learn how to use
nerve impulses, delivered through touch, to create images. Exactly what happens remains
somewhat of a mystery. But more recently, MRI images taken of the brain while it
is working do show the visual cortex of the brain lighting up when receiving sensory
data retrieved through touch.
"The information does get to the area of the brain that is responsible for vision,"
said Kurt Kaczmarek, a UW-Madison engineer and scientist who was involved in the
early work on BrainPort.
The tongue is the perfect organ for the task, Beckman said, because it is moist and
an excellent transmitter of electrical signals, and it has more tactile nerve endings
than any other part of the body except for the lips.
Though one can read the science over and over again, it still requires somewhat of
a leap of faith to grasp the idea of "seeing" through the tongue. Simply, the patterns
of light picked up by the camera are converted by a tiny computer into electrical
pulses across 100 stainless steel electrodes. Users say it feels similar to touching
a weak battery to your tongue, a bubbly or tingling sensation.
The pulses are spatially encoded, meaning the person receiving those signals on the
tongue can perceive depth, perspective, size and shape. That information is translated
by the brain into images — fuzzy images, because of the low resolution, but images
nonetheless. Those who have used the device explain that they perceive the objects
in front of them, separate from their own bodies.
A milestone of sorts
Weihenmayer recalled how when he first tried BrainPort, the researchers sat him down
at a table, fitted him with the device, and then rolled a ball toward him.
"It’s a hard thing to wrap your brain around," said Weihenmayer. "But when they rolled
a white tennis ball toward me, I could feel the ball rolling. First I could feel
the ball starting at the back of my tongue and getting bigger and bigger, coming
toward me. And then I reached out and grabbed it."
When he ascends a rock climbing wall with BrainPort, Weihenmayer said, he can see
the handholds, their differences in shape and the contrast in light between them
and the background. What he sees, he explained, is largely shapes and light variations,
sort of an out-of-focus image.
Last month, Weihenmayer joined Beckman at the National Eye Institute’s 40th anniversary
celebration to demonstrate BrainPort and some of its powers. It seemed a milestone
of sorts.
But the man whose genius led to the creation of such a useful invention was not present.
Bach-y-Rita died of cancer in November of 2006.
"He would have loved to have been there," said Beckman.