Building Emotion-Sensitive Robots

Jan. 3, 2003 -- The line between man and machine is becoming less distinct as technology evolves.

Researchers at the Vanderbilt University in Nashville, Tenn., are working to develop a robot that can respond to human emotions.

Nilanjan Sarkar, an assistant professor of mechanical engineering and the principal researcher for the robot project, says the prime motivation is to develop a better, more natural means for machines to interact with humans.

"There is a lot of communication that is implicit between two persons," says Sarkar. "We study each other's faces and body language to see how the other person reacts. 'Are you bored, are you paying attention, are you excited?' "

But to pick up on these human cues, Sarkar proposes taking a different tact from those tried by other robotics researchers. Rather than using cameras to capture visual clues, Sarkar's research focuses on the "physiological aspects" of human expressions.

"For example, if you are excited, your heart beats faster, your palms might get sweaty," says Sarkar.

Signs of Stress

So Sarkar worked with Craig Smith, an assistant professor of psychology at Vanderbilt, to figure out how to track and translate those physiological changes.

In their research, Sarkar and Smith outfitted human volunteers with sensors that measured certain biological aspects, including heartbeat, facial muscle movement and hand sweat.

To capture physiological data associated with stress and anxiety, they monitored the volunteers while they played video games. The data was then analyzed and translated into algorithms, or "rules," that could be used by a computer processor.

The digital rules were then placed inside a small, wheeled robot that monitored the volunteers' physiological sensors through wireless links. The autonomous robot roamed the labs but when it "sensed" a certain level of stress from a volunteer, it would return to a predetermined spot and ask if it could be of assistance.

Sarkar says the preliminary concepts and results, reported in last month's Robitica journal, published by Cambridge University Press in England, have been promising. And the research, once fully developed, could lead to interesting uses.

"There are many situations where this would be ideal," says Sarkar. "In rehabilitation where a patient has to relearn the use of a limb, it can get quite frustrating and they quit. If you have a robotic aid that senses that frustration, it could help them along the way — modifying the rehabilitation, much like a personalized teacher."

Working Together

One of the biggest applications, however, could be for space-based robots. In fact, a portion of the Vanderbilt research was funded by the NASA Institute for Advanced Concepts in Atlanta.

"Anytime you have humans in space and robots in space, you would like them to cooperate working together," says Robert Cassanova, director of NIAC. "If humans can seamlessly integrate with robots, without the use of joysticks or other interfaces, it would make things better."

Cassanova says NIAC gave the Vanderbilt team about $75,000 to develop and test the system last year. But he wouldn't disclose the test results since Sarkar's research is currently under review by NIAC peers.

But even if NIAC and other scientists approve of the emotion-sensing system, many — including Sarkar — say that the work is still very much in its preliminary stages.

For one, Sarkar notes that the system still has to be able to detect "emotional states" other than stress and anxiety. And developing those algorithms can be tricky, since the physiological distinctions among certain emotions still needs to be understood.

NIAC's Cassanova said the Vanderbilt team's research is one of many being considered for additional funds this year. If the peer review is positive, Sarkar and his team could get as much as $500,000 for additional research.

But even then, Cassanova doubts that even NASA would see emotionally sensitive robots any time soon.

"Generally, we fund concepts that are very futuristic and leaps ahead of anything now," says Cassanova. "These are grand ideas that won't appear for 10 or 40 years into the future."