Using AI for early dementia diagnosis and prediction

How does AI process audio?

  • human voices
  • animal sounds
  • machine noises
  • ambient sounds

HOW DOES AI AUDIO PROCESSING APPLY TO DEMENTIA DIAGNOSIS?

How do you train AI to analyze dementia patterns?

  • Speech-representation neural network which accounts for extracting speech features (embeddings), and
  • Classification model which learns patterns from feature-extractor output
Cookie theft graphic task for dementia diagnosis

ANALYZING DATA

Audio length distribution in DementiaBank dataset

DATA PREPROCESSING

APPROACHES TO SPEECH CLASSIFICATION

  1. Converting speech to text, and using text as an input for the classification model training.
  2. Extracting high-level speech representations to conduct classification on them. This approach is an end-to-end solution, since audio data doesn’t require conversion into other formats.

Modeling

  • Recall evaluates the fraction of correctly classified audio records of all audio records in the dataset. In other words, recall shows the number of records our model classified as dementia.
  • Precision metric indicates how many of those records classified with dementia are actually true.

APPROACH 1: TEXT-TO-SPEECH IN DEMENTIA CLASSIFICATION

End-to-end dementia classification with AI
  1. Converting audio speech into text basically means losing information about the pitch, pauses, and loudness. Once we extract the text, there is no way feature extractors can convey this information, while it’s meaningful to consider pauses during the dementia classification.
  2. The second reason is that the BERT model uses predefined vocabulary to convert word sequences into tokens. Depending on the quality of recording, the model can lose the information it’s unable to recognize. This leads to omission of, for example, incorrect words that still make sense to the prediction results.

APPROACH 2: END-TO-END PROCESSING

Dementia models’ inference results

How to improve the results?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
MobiDev

MobiDev

We create complex business-driven solutions, with a focus on innovation. We write about trends and expertise in AI, IoT, AR & more.