New app helps people with neurological conditions practise speech

Researchers at CSIRO and University of Queensland have developed an app called Harlie that is designed to chat with humans. Unlike Siri and Google Now – which are designed to answer specific questions – Harlie is designed for small talk. Its purpose is to help the user and health professionals understand the impact of neurological conditions on communication and well-being.

People with neurological conditions such as Parkinson’s disease and dementia can have trouble finding the right words and may lose track of their message mid-sentence.

Users of the app have a conversation with the chat-bot, which then analyses aspects of the health of the user’s voice and communication. This includes how well vowels are articulated, vocabulary and duration of mid-sentence pauses.

People who might be working on improving their voice or communication due to difficulties related to a neurological condition, like Parkinson’s disease or stroke, may need to practise and get feedback daily. Conversation with a partner is much more motivating than voice exercises and means people are practising in context too.

But not everyone has easy access to a conversation partner. People increasingly live alone and may have difficulty accessing the community.

Concerns about a future shortage of formal and informal carers are also growing. When people are working on their communication, they may lack confidence and prefer an unrushed and nonjudgmental conversation partner.

As well as offering someone to practise speech with, the app records their speech. This provides objective information about the user’s voice and communication to their health teams.

Change in vocal qualities or conversation patterns can show the impact of speech and language rehabilitation treatments, encouraging people to keep working on their voice. This also indicates the positive and negative impact of medications and other treatments.

Getting treatment dosages exactly right to minimise symptoms, while not leading to side effects, is one of the biggest challenges for people living with Parkinson’s disease. This additional information could help optimise medical treatments. Large changes to communication abilities could also indicate the need for a review by the healthcare team.

Changes in conversation, such as increased difficulty with finding the right word or following the conversation, could indicate a change in cognition and language. It could also indicate the user is still communicating well, recalls recent memories and is thus less likely to need immediate intervention.

With respect to Parkinson’s disease, it could tell the researchers how well their speech rehabilitation is going – for instance, if they are speaking more loudly and articulating words much more clearly.

A demonstration of how the app works can be seen in this YouTube video:

A demonstration of Harlie the chat-bot.

Research shows people may have more problems with communication than they realise. Undiagnosed communication issues might lead to frustration, embarrassment and social withdrawal. These issues all place strain on the person and on their relationships with family members and caregivers.

Technology can help in two ways. First, technology that can monitor progress and the impact of difficulties with communication could help the person, as well as health professionals and researchers, understand the impact of health conditions on communication.

Second, technology can provide people with a way to privately practise their communication without feeling frustrated or judged, while receiving encouraging and productive feedback.

Various technologies help people with Parkinson’s disease with walking, eating and remembering medication. There has been less focus on communication and speech, and using small talk on a smartphone to monitor and help is a new innovation.

Being able to use everyday technology like smartphones makes the technology easier to access, cheaper and more acceptable than specialised medical equipment.

We would like people from the general public to chat to Harlie so we can understand how well it engages in conversation and copes with language, particularly Australian slang, and improve its repertoire of topics. In doing so, we can better prepare Harlie for more clinical studies. We hope this will improve our understanding of health conditions and keep people communicating.