Current voice-activated technology used in virtual assistants is limited by its inability to decipher human emotions, causing it to provide irrelevant responses or miss the point of some conversations entirely.
But a team of researchers from RMIT University’s School of Engineering led by Associate Professor Margaret Lech has discovered how to add emotional capabilities to machines to make communication more natural and more socially acceptable.
“There’s always an emotional context when we talk to people and we understand it, but machines don’t understand this,” says Lech.
“When we call an automatic call centre, for example, people get very frustrated because they talk to the machine and it does not understand that they are sad, they are anxious, that they want things to be done quickly.
“They don’t understand the emotion associated with the task and we can hear from many recordings people saying, ‘I want to talk to a person, not a machine’.
“There is no way to explain certain things to a machine, including those subtle cues that we can express through emotions when we talk to each other.”
Lech and her team have spent 11 years creating new machine learning techniques that allow technology to understand human emotions from speech signals, and to analyse and predict patterns of emotional interactions in human conversations.
Equipped with these capabilities, voice-activated devices can now understand both the linguistic and emotional contents of speech, and provide appropriate responses. They can read seven human emotions: anger, boredom, disgust, fear, happiness, sadness and neutral.
The challenge of making machines read human emotions lay in measuring the unspoken commands in voices such as subtle changes in tone, volume and speed.