Daily Bulletin

The Conversation

  • Written by The Conversation
imageIt's written on your face.Shutterstock

Hearing aids can be lifelines for people with hearing loss. But their limitations can mean that, in particularly noisy environments, users cannot exploit the best of the existing technology. Most new hearing aid designs just make small improvements to microphones, power efficiency and noise filtering. We propose an entirely new approach.

My colleagues and I are working as part of a multi-disciplinary team led by Stirling University, which includes a psychologist and a clinical scientist and is supported by a hearing aid manufacturer. Our aim is to develop an audio-visual hearing aid for the 21st century, taking inspiration from the way that the human body naturally deals with noisy situations, something often known as the cocktail party effect.

Imagine a scenario such as a very busy party with lots of noise, music and people talking. Despite this overwhelming environment, a person with full hearing is often able to pick out and listen to the voice of someone next to them. This is something that people with hearing aids often find extremely challenging. In fact, in really busy environments many deaf people may prefer to remove their hearing aids altogether.

Dealing with noisy environments

The answer to why it is so difficult for hearing aids to deal with these situations is complicated. It’s partly down to the limitations of directional microphones, of inadequate noise cancelling, and of the loss of information about where sound is coming from. But the reason why deaf people can often “hear” better in overwhelming environments like this can be partly explained by lip-reading.

Lip-reading is known to enable individuals with hearing-loss to better understand speech. We all lip-read to a greater or lesser extent, but in people with hearing loss it can become a vital skill. Yet it’s a component of communication that existing hearing aids simply ignore.

imageNo more smiling and nodding.Shutterstock

Our vision is for an ear or body-worn hearing aid linked to a small wearable camera, which could be mounted in a pair of ordinary glasses, jewellery or perhaps even worn as a discreet badge. The device would process the camera’s video stream to isolate relevant information about lip movement.

This data can be used by the hearing aid in several ways. On a simple level, if it knows someone is speaking it could apply some general background noise-reduction filtering. It could identify the direction the voice is coming from and focus a directional microphone accordingly.

Significantly, it could also use the lip movement information to apply an appropriate filter for further noise reduction, just as our brains do naturally. Specifically, if the device can estimate what the speech is likely to be, then it can remove sound elements that don’t match this. For example, if loud music is playing, “reading” the lips of the target speaker would indicate to the device that it should remove this music because it does not match the expected sound.

What are the challenges?

There are multiple challenges to ensuring a hearing aid like this can work practically in the real world, involving the same problems that human lip-readers face. It has to be able to deal with multiple speakers at once and sound that isn’t in front of it. And, generally, people do not simply stand motionless in front of the listener, but instead tend to move, turn their heads, cover their faces or show their emotions visually. They may also be interrupted or have someone else walk in front of them.

To overcome this, our solution will be to again consider how humans function. How much lip-reading we do depends on the circumstances. The more noisy it is, the more we tend to look at people’s lips. So a system that exclusively lip-reads would not be very useful when it comes to real conversations in real environments. We plan to integrate our approach with other non-camera approaches that hearing aids presently use, including noise cancelling and directional microphones.

Our aim is to produce an aesthetically designed system that improves users' ability to understand what someone is saying in a range of environments, potentially with less listening effort. This would help solve the real problems faced by those with hearing loss, including their low uptake of available technology, by delivering a freely available, next-generation hearing device prototype, inspired by the way we naturally think, hear and see.

Amir Hussain receives funding from the EPSRC and Royal Society of Edinburgh.

Authors: The Conversation

Read more http://theconversation.com/our-lip-reading-technology-promises-to-make-hearing-aids-more-human-45166

Writers Wanted

The big barriers to global vaccination: patent rights, national self-interest and the wealth gap


After riots, Donald Trump leaves office with under 40% approval


Five ways Australians can save the planet without lifting a finger (well, almost!)


The Conversation


Ray Hadley's interview with Scott Morrison

RAY HADLEY: Prime Minister, good morning.    PRIME MINISTER: G’day Ray.   HADLEY: I was just referring to this story from the Courier Mail, which you’ve probably caught up with today about t...

Ray Hadley & Scott Morrison - avatar Ray Hadley & Scott Morrison

Prime Minister's Remarks to Joint Party Room

PRIME MINISTER: Well, it is great to be back in the party room, the joint party room. It’s great to have everybody back here. It’s great to officially welcome Garth who joins us. Welcome, Garth...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Ben Fordham, 2GB

BEN FORDHAM: Scott Morrison, good morning to you.    PRIME MINISTER: Good morning, Ben. How are you?    FORDHAM: Good. How many days have you got to go?   PRIME MINISTER: I've got another we...

Scott Morrison - avatar Scott Morrison

Business News

7 foolproof tips for bidding successfully at a property auction

Auctions can be beneficial for prospective buyers, as they are transparent and fair. If you reach the limit you are willing to pay, you can simply walk away. Another benefit of an auction is tha...

Dominique Grubisa - avatar Dominique Grubisa

Getting Ready to Code? These Popular and Easy Programming Languages Can Get You Started

According to HOLP (History Encyclopedia of Programing Languages), there are more than 8,000 programming languages, some dating as far back as the 18th century. Although there might be as many pr...

News Co - avatar News Co

Avoid These Mistakes When Changing up Your Executive Career

Switching up industries is a valid move at any stage in your career, even if you’re an executive. Doing so at this stage can be a lot more intimidating, however, and it can be quite difficult know...

News Co - avatar News Co

News Co Media Group

Content & Technology Connecting Global Audiences

More Information - Less Opinion