How we detect cough

Machine learning is a set of methods that are used to gain insights and discover patterns in a given set of data, known as training data, and then using those insights to make predictions on new data. Machine learning takes advantage of the fact that computers are excellent at processing very large amounts of data quickly and can “see” small differences that humans cannot detect. Engineers use these abilities to design algorithms to detect these differences and sort data into categories (“This sound is a cough while that sound is not”).

Hyfe uses machine learning algorithms to detect when a sound is a cough. As in all machine learning, choices made when building the model have important implications for how it works later on. When we designed the algorithm in CoughPro to detect cough, among the considerations that we prioritized were: privacy, computing, memory, and bandwidth usage, as well as the availability of data to train the algorithm to do its job.

The state-of-the-art machine learning models in CoughPro can identify if a short snippet of audio contains a cough within milliseconds.

The CoughPro algorithm converts audio signals sampled by smartphone into a visual representation that shows the frequency of the signal as it changes over time. This visualization is called a “spectrogram.” This helps the algorithm identify patterns in the training data that are only present in cough sounds – for example relatively higher energy at specific frequencies.

A visual representation of the audio signal is transformed multiple times and translated into parameters inside a neural network. This way the neural network “learns” the correct set of weights that can accurately predict if a sound is a cough.

We use training data that contain both cough sounds as well as non-cough sounds. This is critical for the neural network to identify audio signals correctly by differentiating them from non-cough sounds. This enables the algorithm to identify patterns in a cough that are universal.

The most challenging part of detecting coughs is building an algorithm that works universally. It’s very easy to build an algorithm that performs well on the data seen during training but fails miserably when faced with something new. This may seem boring but it is important work - to validate that the algorithm works on future as well as already known data.

At Hyfe, we take a lot of care to ensure that the algorithm built into CoughPro works for everyone – whether you’re at a retirement home or trying to track your toddler’s coughs. This is accomplished in two ways: by building the model and training it on a large, heterogeneous dataset, and by using machine learning techniques that prevent the algorithm from focusing too much on patterns that are only present in the training data (i.e., “overfitting”).

Coughs don’t sound the same everywhere - if you ask someone to cough voluntarily, it might sound very different from a spontaneous cough. To learn different types of cough sounds, we collect training data in multiple conditions that reflect our users’ lives – in quiet locations as well as noisy, challenging locations like construction sites or covid care centers (multiple people coughing). We also collect coughs across many varied demographic categories (age, gender, geography) to ensure that the model can recognize coughs no matter who is doing the coughing.