Create Profile

Creating your profile will enable you to submit photos and stories to get published on News24.

Please provide a username for your profile page:

This username must be unique, cannot be edited and will be used in the URL to your profile page across the entire 24.com network.

Facebook Sign-In

Hi News addict,

Join the News24 Community to be involved in breaking the news.

Log in with Facebook to comment and personalise news, weather and listings.


Smartphones to develop emotional intelligence

04 December 2012, 11:12

If you think having your phone identify the nearest bus stop is cool, wait until it identifies your mood.

New research by a team of engineers at the University of Rochester may soon make that possible.

At the IEEE Workshop on Spoken Language Technology, the researchers will describe a new computer program that gauges human feelings through speech, with substantially greater accuracy than existing approaches.

Surprisingly, the program doesn't look at the meaning of the words.

"We actually used recordings of actors reading out the date of the month – it really doesn't matter what they say, it's how they're saying it that we're interested in," said Wendi Heinzelman, professor of electrical and computer engineering.

Heinzelman explained that the program analyses 12 features of speech, such as pitch and volume, to identify one of six emotions from a sound recording.

And it achieves 81% accuracy – a significant improvement on earlier studies that achieved only about 55% accuracy.

Prototype of an app developed

The research has already been used to develop a prototype of an app.

 The app displays either a happy or sad face after it records and analyses the user's voice.

 It was built by one of Heinzelman's graduate students, Na Yang, during a summer internship at Microsoft Research.

"The research is still in its early days," Heinzelman added, "but it is easy to envision a more complex app that could use this technology for everything from adjusting the colours displayed on your mobile to playing music fitting to how you're feeling after recording your voice."

Heinzelman and her team are collaborating with Rochester psychologists Melissa Sturge-Apple and Patrick Davies, who are currently studying the interactions between teenagers and their parents.

 "A reliable way of categorizing emotions could be very useful in our research," Sturge-Apple said.

"It would mean that a researcher doesn't have to listen to the conversations and manually input the emotion of different people at different stages."

Teaching a computer to understand emotions begins with recognizing how humans do so.

"You might hear someone speak and think 'oh, he sounds angry!' But what is it that makes you think that?" asks Sturge-Apple.

She explained that emotion affects the way people speak by altering the volume, pitch and even the harmonics of their speech.

"We don't pay attention to these features individually, we have just come to learn what angry sounds like – particularly for people we know," she adds.

But for a computer to categorize emotion it needs to work with measurable quantities. So the researchers established 12 specific features in speech that were measured in each recording at short intervals.

The researchers then categorized each of the recordings and used them to teach the computer program what "sad," "happy," "fearful," "disgusted," or "neutral" sound like.

Speech and emotion

The system then analysed new recordings and tried to determine whether the voice in the recording portrayed any of the known emotions.

If the computer program was unable to decide between two or more emotions, it just left that recording unclassified.

"We want to be confident that when the computer thinks the recorded speech reflects a particular emotion, it is very likely it is indeed portraying this emotion," Heinzelman explained.

Previous research has shown that emotion classification systems are highly speaker dependent; they work much better if the system is trained by the same voice it will analyze.

"This is not ideal for a situation where you want to be able to just run an experiment on a group of people talking and interacting, like the parents and teenagers we work with," Sturge-Apple explained.

Their new results also confirm this finding. If the speech-based emotion classification is used on a voice different from the one that trained the system, the accuracy dropped from 81% to about 30%.

The researchers are now looking at ways of minimising this effect, for example, by training the system with a voice in the same age group and of the same gender.

As Heinzelman said, "there are still challenges to be resolved if we want to use this system in an environment resembling a real-life situation, but we do know that the algorithm we developed is more effective than previous attempts."

(EurekAlert, December 2012)


Read News24’s Comments Policy

Comment on this story
Comments have been closed for this article.

Read more from our Users

Submitted by
Wilson Ochieng
ODM MP chased down by angry Kibra...

Kibra MP Ken Okoth had a hard time in his constituency after angry youth pelted him with stones. Read more...

Submitted by
Wilson Ochieng
Prepare for DP Ruto fight in 2022...

An MP has warned that the Kalenjin Community will not stand back and watch as DP Ruto is duped ahead of the 2022 polls. Read more...

Submitted by
William Korir
Be careful who you deal with, DP ...

Watch out for your political future, DP William Ruto has been warned. Read more...

Submitted by
William Korir
Kikuyu 2022 vote will go to Peter...

The 2022 Kikuyu vote will go to Peter Kenneth, a Rift Valley Governor has warned DP William Ruto. Read more...

Submitted by
William Korir
CORD threatens to boycott 2017 po...

The CORD Coalition says that it will be forced to boycott the 2017 elections if there is no new voter register in place by the time. Read more...

Submitted by
Wilson Ochieng
Disgraced Jeptoo stripped of 2014...

Organizers of the Boston Marathon are stripping Kenyan runner Rita Jeptoo of her 2014 victory as part of the athlete's newly extended doping ban. Read more...