Just cough into your phone, please... MIT lab thinks it can diagnose COVID-19 from the way you expectorate

Academics claim their AI software can detect, with 98.5 per cent accuracy, whether or not someone has caught the COVID-19 coronavirus, just from the sound of their coughing. To build this software, an MIT team used three ResNet50 models, a popular convolutional neural network designed by Microsoft. They’re normally used to …

[Mike 137]

"98.5% accuracy"

Claims like this need robust justification. A sample of five thousand (effectively divided by two of course) seems rather small, and I'd really like to know the false positive and false negative rates. I'd also like to know the number of trials per subject and their distribution as well. Sadly the journal is paywalled so I can't find out. However, quite apart of the honesty of the subjects, I can think of quite a few distorting factors that could contribute to uncertainty.

[Francis Boyle]

Re: "98.5% accuracy"

Agreed. In 2020 no one should be talking about the accuracy of their test without specifying both false positive and false negative rates. (In this case 1.5% is probably fine for the former but not exactly great for the latter.)

[Anonymous Coward]

Re: "98.5% accuracy"

> A sample of five thousand seems rather small

According to the article, the sample size was 1,000, which is perfectly adequate. The rest of the data was used to train the system.

[Anonymous Coward]

Re: "98.5% accuracy"

> I'd really like to know the false positive and false negative rates. [...] Sadly the journal is paywalled so I can't find out.

"When validated with subjects diagnosed using an official test, the model achieves COVID-19 sensitivity of 98.5% with a specificity of 94.2% (AUC: 0.97). For asymptomatic subjects it achieves sensitivity of 100% with a specificity of 83.2%."

It's in the abstract, one click away from the Reg article.

[Anonymous Coward]

I'm catching up with them

The application I'm developing, which makes a diagnosis based on the way you fart, right now has both sensitivity and specifity of 50%.

[Fruit and Nutcase]

Re: I'm catching up with them

See if you can correlate the methane content - then come up with an app that you can offset the carbon automatically.

[Anonymous Coward]

Re: I'm catching up with them

Set it alight, fewer greenhouse gases that way.

[hoola]

Re: I'm catching up with them

But it will make good headlines,

What is probably far more useful and a better investment in cache is the breath test based on a concept developed by Loughborough University.

https://www.lboro.ac.uk/news-events/news/2020/october/covid-breath-test/

Maybe I am just cynical but I would have more faith in a scientific test that coughing into a phone running an App. Now if were then swab the phone and test that then we might have something useful.

There is this endless obsession that somehow technology and an App is the solution to every problem that exists. The fact that time and time again it has proved not to the case does not appear to deter people.

[sabroni]

Re: I would have more faith in a scientific test that coughing into a phone running an App

The disadvantage of the "blowing into a tube" method over the "use your phone" method is that people don't already have those tubes. A reliable phone based covid cough detector is a valuable tool and worth persuing.

Writing it off because apps are trendy seems un-scientific.

[Anonymous Coward]

Re: I'm catching up with them

Surely it should ask how many toilets rolls you have bought. There seems to be a correlation between COVID and toilet roll use.

[Lon24]

Re: "98.5% accuracy"

A normal future has been predicated on a successful vaccine and better treatments. Both of which are far from assured. We need to find solutions outside those boxes.

Here we have a potential game-changer. It's potential is to capture asymptotic cases and direct them into the full test & trace system at no cost. If it can achieve true 95%+ levels of assurance there and if we can combine that fast full test, trace & quarantine response then we should be able to close down the virus to very low levels and maintain that without further significant medical intervention and cost.

The danger is in the number of false positives may overwhelm the test and trace system. Given the billions (trillions?) being put into vaccines throwing a few million at checking this and other approaches out may be very good value.

Not forgetting a 99% effective T&T system. That's the really hard bit and clearly beyond the current UK setup. Something for the WHO to make the sharing of international best practice rise above nationalistic boasting.

[Anonymous Coward]

Re: "98.5% accuracy"

Not forgetting a 99% effective T&T system. That's the really hard bit and clearly beyond the current UK setup.

It's beyond any setup, except maybe in totalitarian states like China or N. Korea.

[Anonymous Coward]

Re: It's beyond any setup

What about Australia, Taiwan, Vietnam, New Zealand and South Korea?

Not completely under control but not the exponential shit show that's occuring in Europe atm.

What's your agenda? Why claim that it's impossible to control? I guess you don't work in health care....

[Anonymous Coward]

Re: "98.5% accuracy"

Here we have a potential game-changer. It's potential is to capture asymptotic cases and direct them into the full test & trace system at no cost.

Whilst it is an interesting result worthy of further investigation, it is not quite so simple. Consider the implications of "To that end, it is remarkable that our tool discriminates 100% of asymptomatics at the expense of a false positive rate of 16.8%."

About 1 in 200 people in the UK have COVID. Hence, if you used a test like that across the whole population, then 70 million * 0.005 = 350,000 people with the illness (including the 40% (140,000) without symptoms) would be sent off for testing. However, a false positive rate of 16.8% would mean that 70 million * 0.995 * 0.168 = 12 million people (i.e. 17% of the UK population) who don't have COVID would also be sent of for testing.

[Cuddles]

Re: "98.5% accuracy"

While true, I think that's looking at it a bit backwards. The goal is to test the minimum number of people to ensure all cases are found. In the absence of any pre-testing, that means testing every single person. So a pre-test with a 17% false positive rate isn't bad because it makes you test 12 million people without an infection, it's good because it means you haven't had to test the other 58 million. Obviously it would be nice if it was even better, but as a first pass requiring viritually zero resources it's really very promising.

[Anonymous Coward]

Re: "98.5% accuracy"

While true, I think that's looking at it a bit backwards. The goal is to test the minimum number of people to ensure all cases are found. In the absence of any pre-testing, that means testing every single person.

I think it means testing every single person whom the test track and trace system has identified as potentially infected.

[Cuddles]

Re: "98.5% accuracy"

I think it means testing every single person whom the test track and trace system has identified as potentially infected.

No. As I said, the goal is to find all cases. In the absence of any other information, that means testing everyone. A track and trace system is simply one way to try to filter the number of people who need testing. Early on it may have been pretty much the only option, but that doesn't mean it's the best, or the most effective use of limited resources.

[Steve K]

Re: "98.5% accuracy"

asymptotic cases

..one hopes that the infection curve does not progress this far...

[Anonymous Coward]

Re: "98.5% accuracy"

> Sadly the journal is paywalled so I can't find out.

What are you talking about? That journal is 100% open access.

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9208795

[jake]

Re: "98.5% accuracy"

All the justification needed is that people are throwing grant money at them, and possibly even start-up money, if they can figure out a way to add blockchain to the final product.

[jake]

Re: "98.5% accuracy"

Our DecalPercentage.

_{Spot the reference, win a beer.}

[FlamingDeath]

Re: "98.5% accuracy"

Shhhhhh, stop asking stupid questions, looks, it’s Ai, ooooooh shiney, so futuristic

This dude, is selling a product, and you’re cramping their style

[Sorry that handle is already taken.]

"...from the way you expectorate"

I'm not spitting into my phone.

[Neil Barnes]

Re: "...from the way you expectorate"

But it's better to expectorate than never...

[pavel.petrman]

Re: "...from the way you expectorate"

Expectorate Patronum!

[jake]

Re: "...from the way you expectorate"

You "spit" into your phone every time you talk into it.

Oh, wait, what am I saying ... talking on the phone is SO 20th Century.

[I ain't Spartacus]

Re: "...from the way you expectorate"

No-one expectorates the Spanish Inquisition!

[Danny 2]

Every silver lining has a cloud (of aerosol)

I'd like to test that app, I've had a terrible cough since November. I mean it's been bad since 1985 but the last year it's been constant and painful. I know, I should see a doctor especially since I read that article by the young man who died of a brain tumour. Still, I don't want to go to the doctors during the pandemic in case it's nothing, and if it is something I'd rather not know. If it's still bad in a month then I'll cut back on my smoking. Meanwhile it's a useful way to get people to stand two metres away from me in shops.

The uni's AutoID Lab earlier developed a speech-analyzing model to screen people for Alzheimer's testing, though turned their attention to COVID-19 as the pandemic took hold.

That's actually true and obvious - my dad has dementia - but more so is the 'dementia shuffle', a characteristic walk without raising their feet. I suggested buying him 'wheelies' trainers but my mum opted for a wheeled zimmer frame.

[jake]

Re: Every silver lining has a cloud (of aerosol)

"I don't want to go to the doctors during the pandemic in case it's nothing"

That's a piss-poor excuse at the best of times .... however, it doesn't answer the question of why you didn't go to the doctor sometime between November and March, before the pandemic became a known issue ...

[sgp]

Re: Every silver lining has a cloud (of aerosol)

"If it's still bad in a month then I'll cut back on my smoking."

Why not now?

[Danny 2]

Re: Every silver lining has a cloud (of aerosol)

I was joking. If my cough gets worse then I'll have to cut back on the cigarettes but it won't be by choice. I'm twenty years older than I expected to be, those two decades have been okay but the next two I could pass on happily.

[Danny 2]

Re: Every silver lining has a cloud (of aerosol)

Jake,

I was convinced of the inevitably of a serious pandemic at the start of January, but I was also subject to other factors. The Scottish winter, the ill health of my parents, the fact to see a doctor I have to wait in line for two hours outside my health clinic, then sit inside for another two hours alongside undeniably sick folk often with coughs as bad as, or worse than, mine.

Plus who do you think I have to excuse myself to? My grandpa taught me these lyrics to live and die for:

I'll eat when I'm hungry, I'll drink when I'm dry,

If the moonshine don't kill me then I'll live 'til I die

[chivo243]

Medication

My meds specifically mention stuffy and or runny nose, and dry cough as side effects, I have both now and then. Does this study\app take these conditions into consideration?

[Mathman]

Data set

I wonder if the dataset included people who had other infections such as coughs colds and flu. Wouldn't be surprised if the input was people who tested positive and people who who have no symptoms. So it may be no better than saying have you got a persistent cough? Also those who know they have Covid may be recording under different conditions (physical and mental). So it would be important to test on a random set of subjects and only then test them for Covid with a follow up test a week later.

[Anonymous Coward]

Re: Data set

Smokers/Non-smokers.

[Anonymous Coward]

Expectorate! Expectorate!

Souhds like a Dominic Cummings-flavoured Dalek.

[pavel.petrman]

Re: Expectorate! Expectorate!

- Why don't they just say spit?

- Do you want them to save time?

[Anonymous Coward]

Re: Expectorate! Expectorate!

Boris Johnson's Commemoration Stamps featuring Dominic Cummings will prove an abysmal failure.

[DS999]

This needs to be an app, not a "device"

Put it into a device and we won't see it until the pandemic is over. Put it in an app and everyone can have it in weeks, and could test themselves if they are concerned if their cough is a problem.

98.5% is FAR more effective than the temperature checks some places do before admitting people, it would be easy to have a (well protected) person stationed outside having people cough into a smartphone and let the app decide if they can enter or not. Sanitize the phone, repeat for the next person.

[cbars]

Useless

Discerning a forced cough and COVID is as useful as asking, have you got a cough? But really, have you?

Anyone with a cold/other cough in the sample...?

[Anonymous Coward]

Re: Useless

Came here to make the same point about the forced cough but the survey is even more bollocks (or wondrous, depending on your pov) because the recordings weren't made when the person had Covid. So what the researchers have managed to produce* is an app that can distinguish between the forced cough of someone who tested positive for Covid an unknown numbers of days or weeks earlier, versus a forced cough from someone who tested negative for Covid an unknown number of days or weeks earlier.

_{[*] According to the information in the article - I'm not going to spoil my indignation by actually reading the original to check!}

[Mike 125]

Re: Useless

>I'm not going to spoil my indignation by actually reading the original to check!

Yea, I'm generally going with that too...

"For asymptomatic subjects it achieves sensitivity of 100% with a specificity of 83.2%."

'asymptomatic' adjective, definition: (of a condition or a person) producing or showing no symptoms.

Anyone else see a problem here..?

I call bullsh't, (but hope to be wrong.)

[LateAgain]

Not hard to check

Use it at a testing centre and note the result on the samples sent off.

It shouldn't take long to see how closely they match.

[Anonymous Coward]

pointless

The whole exercise is pointless

You are at your most infectious 24 hours *before* you show any symptoms at all.

So quite apart from the large number of symptomless cases this won't do much to reduce transmission of the virus even if it works perfectly.

The Loughborough/IMSPEX/Edinburgh/Dortmund breath test seems like a much more promising system.

[Luigi D'Goomba]

Phone placement is everything

And if a male happens to place his phone on his nutsack and turn his head while he coughs, he can get a two-fer test for COVID and hernia.

Thank you very much ladies and gentlemen, I'm here 'til Thursday.

[Steve K]

Re: Phone placement is everything

and that's bollocks too.....

[CrackedNoggin]

MIT does fantastic work - but there is a lot of pressure and sometimes passionate optimistic fervor jumps the constraints - Cold Fusion, Food computers@Media Lab. I hope that is not the case here. I'm with holding judgement until the claim is properly validated by third party research.

That it can attract invest money at this stage is actually not a good sign in my opinion.

The people who would get hurt the most if this flops are the researchers involved.

[FrenchFries!]

Cystic Fibrosis

Rather fascinating. I'm going to try to look at this objectively.

My fiance' had Cystic Fibrosis. You can "generally" tell by by how they cough that they have CF.

I know this is anecdotal and baseless from a data standpoint, but, the more we know, and the different approaches to knowing and understanding, hopefully help.

[Anonymous Coward]

Finall!y!

A good use for wiretapping the entire population...

[Anonymous Coward]

Coming soon ?

"Alexa. Covid check. <cough> <cough>"

[Robert Carnegie]

Just thinking

Who remembers Theranos pivoting...

I didn't - I got it mixed up with "Theramed". Which is toothpaste and mouthwash apparently. Theranos was eyewash.

[Richard Pennington 1]

The late, great Douglas Adams predicted ...

... telephone sanitisers.

I'll get my towel.

[Robert Carnegie]

Douglas Adams apparently did not invent the professional telephone sanitiser.

http://www.guter.org/bristow/pretty.htm

refers to a comicstrip character older than me, the lady from Kleenaphone who would visit the fictional monolithic office where "Bristow" worked. According to the article, readers never saw Miss Pretty, she came and went, but did encounter her colleague Miss Dimmitty.

Douglas Adams's list of enemies of the people* also included hairdressers and we may be thinking twice about that now. I think the situation may have been that he didn't particularly care about the state of his hair, and he projected that onto hair being generally irrelevant to everybody.

*The people of the planet Golgafrincham, and the enemies were sent off. Guess to where (did I explain this already?)