Thanks to ASU alumnus Shantanu Bala, a smile can now be felt, rather than just seen.
Bala helped lead an initiative by ASU’s Center for Cognitive Ubiquitous Computing to innovate aids for social interaction in disabled people, primarily those who are blind and deaf.
The results of this research allowed Bala to develop an innovative chair, which allows a blind or visually impaired person to feel someone smile at them through vibrations on the back.
Essentially, the device uses cameras to pick up on facial movements of people in the area of the person using the device. The device would then take the visual stimulation and transfer it over to the series of motor vibrators on the user’s back, said TroyMcDaniel, associate director of CUbiC.
The device would pick up certain facial expressions and vibrate in that shape onto their skin. It would be similar to a back massage but with more of a Professor Quirrell-like twist.
“The field of research that we’re working in, it’s really sensory substitution,” McDaniel said, “It’s this idea that if one of our senses is missing we can present information to a different sense. So for example, for individuals who are blind, we can present visual information through the sense of touch.”
McDaniel served as a mentor for Bala when they both began working with CUbiC roughly around the same time six years ago. Bala was a high school student and McDaniel was a graduate student.
WhenCUbiC first was organized, they had discussed the idea of developing socially assisted aids for the disabled, but the project was still in its infancy when Bala joined. However, Bala eagerly jumped on the idea and decided to use cross-modal technology to assist in this initiative.
“I think that it’s important primarily because technology can enable people to accomplish more than they would be able to otherwise,” Bala said. “This is especially valuable to people who have a disability and are otherwise marginalized and this is a way to ease the process of integrating with a society of people who are able to see or able to hear or don’t really have a perspective of what that’s like to have a disability.”
Bala developed a glove as the project's original prototype, allowing this information to be transferred unto the top of the user's hand.
“We decided to go with the back of the hand because it’s a portion of the body that’s sensitive to touch," Bala said. "If anything touches your hand, your body’s able to pick up on that with much greater detail than almost anywhere else on your skin."
After the glove's development, Bala and CUbiC members began expanding their reach, looking to become more detailed and expand to other horizons, building the chair to provide more detail in the facial expressions and emotions in interactions.
“We can now do things like extract facial action units — these are the basic building blocks of expressions," McDaniel said. "Let's say someone does the simple act of smiling — there’s a lot going on there, lip curls and cheek puffs, you can look at the face and extract many building blocks."
After the chair, however, Bala began exploring new realms: converting hearing into touch using the same technology— a project Bala is currently working on.
He said the deaf, hearing disabled and those using cochlear implants often have trouble sensing tone, emotional ranges in the voice or even the sex of the person they are talking to. So he hopes to use these vibrations to convey pitch by moving them up and down the back.
Each process has had its fair share of hurdles, and those working on the project struggled with how to transfer one set of senses to a completely other medium without changing the entire meaning.
Yet despite these struggles, Bala said he has high hopes for the future of this technology, looking to expand it to a more portable, everyday level.
“I think the biggest next step as far as the devices is to see the extent to which you can provide the most detailed information possible while also integrating those devices with smart phones and other tools people already use on a regular basis," he said.
Reach the reporter at megan.janetsky@asu.edu or follow @meganjanetsky on Twitter.
Like The State Press on Facebook and follow @statepress on Twitter.
