How we detect cough
Machine Learning 101
Machine learning is a set of methods that are used to gain insights and discover patterns in a given set of data, known as training data, and then using those insights to make predictions on new data. Machine learning takes advantage of the fact that computers are excellent at processing very large amounts of data quickly and can “see” small differences that humans cannot detect. Engineers use these abilities to design algorithms to detect these differences and sort data into categories (“This sound is a cough while that sound is not”).
Hyfe uses machine learning algorithms to detect when a sound is a cough. As in all machine learning, choices made when building the model have important implications for how it works later on. When we designed the algorithm in CoughPro to detect cough, among the considerations that we prioritized were: privacy, computing, memory, and bandwidth usage, as well as the availability of data to train the algorithm to do its job.
The state-of-the-art machine learning models in CoughPro can identify if a short snippet of audio contains a cough within milliseconds.
Building and Training Algorithms
The CoughPro algorithm converts audio signals sampled by smartphone into a visual representation that shows the frequency of the signal as it changes over time. This visualization is called a “spectrogram.” This helps the algorithm identify patterns in the training data that are only present in cough sounds – for example relatively higher energy at specific frequencies.
A visual representation of the audio signal is transformed multiple times and translated into parameters inside a neural network. This way the neural network “learns” the correct set of weights that can accurately predict if a sound is a cough.
We use training data that contain both cough sounds as well as non-cough sounds. This is critical for the neural network to identify audio signals correctly by differentiating them from non-cough sounds. This enables the algorithm to identify patterns in a cough that are universal.
A Universal Algorithm
The most challenging part of detecting coughs is building an algorithm that works universally. It’s very easy to build an algorithm that performs well on the data seen during training but fails miserably when faced with something new. This may seem boring but it is important work - to validate that the algorithm works on future as well as already known data.
At Hyfe, we take a lot of care to ensure that the algorithm built into CoughPro works for everyone – whether you’re at a retirement home or trying to track your toddler’s coughs. This is accomplished in two ways: by building the model and training it on a large, heterogeneous dataset, and by using machine learning techniques that prevent the algorithm from focusing too much on patterns that are only present in the training data (i.e., “overfitting”).
Coughs don’t sound the same everywhere - if you ask someone to cough voluntarily, it might sound very different from a spontaneous cough. To learn different types of cough sounds, we collect training data in multiple conditions that reflect our users’ lives – in quiet locations as well as noisy, challenging locations like construction sites or covid care centers (multiple people coughing). We also collect coughs across many varied demographic categories (age, gender, geography) to ensure that the model can recognize coughs no matter who is doing the coughing.
Accuracy
As they say “no model is perfect, but some models are useful.” The CoughPro algorithm can differentiate between a cough and non-cough sound with >99% accuracy. There are still occasionally “false positives” (a non-cough sound that the model incorrectly categorizes as a cough) and “false negatives” (a cough occurs and the model doesn’t catch it). Some degree of inaccuracy is typical of event counters (step counters, heart rate monitors, etc.), but Hyfe’s accuracy level is more than sufficient to provide meaningful trends. CoughPro is extremely accurate in clean acoustic environments and when kept unobstructed, near to the user; in challenging acoustic environments (with lots of background noise, for example), accuracy is reduced.
Hyfe engineers are constantly working on improving accuracy by: (a) re-training the model with new sounds; and (b) experimenting with novel methods for modeling and data processing. Over time, CoughPro will continue to improve. To achieve highest accuracy with CoughPro, users should aim to keep the device nearby, ensure that the microphone is unobstructed, and consider avoiding or pausing tracking while in extremely difficult acoustic environments. For more information on accuracy, consider reading:
- this article on the effect of distance: https://www.hyfe.ai/publication/effects-of-distance-on-hyfe-performance and
- this summary of a validation trial: https://www.hyfe.ai/publication/cough-detection-classificatio