back to article Just cough into your phone, please... MIT lab thinks it can diagnose COVID-19 from the way you expectorate

Academics claim their AI software can detect, with 98.5 per cent accuracy, whether or not someone has caught the COVID-19 coronavirus, just from the sound of their coughing. To build this software, the MIT team used three ResNet50 models, a popular convolutional neural network designed by Microsoft. They’re normally used to …

  1. Mike 137 Silver badge

    "98.5% accuracy"

    Claims like this need robust justification. A sample of five thousand (effectively divided by two of course) seems rather small, and I'd really like to know the false positive and false negative rates. I'd also like to know the number of trials per subject and their distribution as well. Sadly the journal is paywalled so I can't find out. However, quite apart of the honesty of the subjects, I can think of quite a few distorting factors that could contribute to uncertainty.

    1. Francis Boyle

      Re: "98.5% accuracy"

      Agreed. In 2020 no one should be talking about the accuracy of their test without specifying both false positive and false negative rates. (In this case 1.5% is probably fine for the former but not exactly great for the latter.)

    2. Anonymous Coward
      Anonymous Coward

      Re: "98.5% accuracy"

      > A sample of five thousand seems rather small

      According to the article, the sample size was 1,000, which is perfectly adequate. The rest of the data was used to train the system.

    3. Anonymous Coward
      Anonymous Coward

      Re: "98.5% accuracy"

      > I'd really like to know the false positive and false negative rates. [...] Sadly the journal is paywalled so I can't find out.

      "When validated with subjects diagnosed using an official test, the model achieves COVID-19 sensitivity of 98.5% with a specificity of 94.2% (AUC: 0.97). For asymptomatic subjects it achieves sensitivity of 100% with a specificity of 83.2%."

      It's in the abstract, one click away from the Reg article.

      1. Anonymous Coward
        Anonymous Coward

        I'm catching up with them

        The application I'm developing, which makes a diagnosis based on the way you fart, right now has both sensitivity and specifity of 50%.

        1. Fruit and Nutcase Silver badge
          Thumb Up

          Re: I'm catching up with them

          See if you can correlate the methane content - then come up with an app that you can offset the carbon automatically.

          1. Anonymous Coward
            Anonymous Coward

            Re: I'm catching up with them

            Set it alight, fewer greenhouse gases that way.

            1. hoola Silver badge

              Re: I'm catching up with them

              But it will make good headlines,

              What is probably far more useful and a better investment in cache is the breath test based on a concept developed by Loughborough University.

              https://www.lboro.ac.uk/news-events/news/2020/october/covid-breath-test/

              Maybe I am just cynical but I would have more faith in a scientific test that coughing into a phone running an App. Now if were then swab the phone and test that then we might have something useful.

              There is this endless obsession that somehow technology and an App is the solution to every problem that exists. The fact that time and time again it has proved not to the case does not appear to deter people.

              1. sabroni Silver badge

                Re: I would have more faith in a scientific test that coughing into a phone running an App

                The disadvantage of the "blowing into a tube" method over the "use your phone" method is that people don't already have those tubes. A reliable phone based covid cough detector is a valuable tool and worth persuing.

                Writing it off because apps are trendy seems un-scientific.

        2. Anonymous Coward
          Anonymous Coward

          Re: I'm catching up with them

          Surely it should ask how many toilets rolls you have bought. There seems to be a correlation between COVID and toilet roll use.

      2. Lon24 Silver badge

        Re: "98.5% accuracy"

        A normal future has been predicated on a successful vaccine and better treatments. Both of which are far from assured. We need to find solutions outside those boxes.

        Here we have a potential game-changer. It's potential is to capture asymptotic cases and direct them into the full test & trace system at no cost. If it can achieve true 95%+ levels of assurance there and if we can combine that fast full test, trace & quarantine response then we should be able to close down the virus to very low levels and maintain that without further significant medical intervention and cost.

        The danger is in the number of false positives may overwhelm the test and trace system. Given the billions (trillions?) being put into vaccines throwing a few million at checking this and other approaches out may be very good value.

        Not forgetting a 99% effective T&T system. That's the really hard bit and clearly beyond the current UK setup. Something for the WHO to make the sharing of international best practice rise above nationalistic boasting.

        1. Anonymous Coward
          Anonymous Coward

          Re: "98.5% accuracy"

          Not forgetting a 99% effective T&T system. That's the really hard bit and clearly beyond the current UK setup.

          It's beyond any setup, except maybe in totalitarian states like China or N. Korea.

          1. Anonymous Coward
            Anonymous Coward

            Re: It's beyond any setup

            What about Australia, Taiwan, Vietnam, New Zealand and South Korea?

            Not completely under control but not the exponential shit show that's occuring in Europe atm.

            What's your agenda? Why claim that it's impossible to control? I guess you don't work in health care....

        2. Anonymous Coward
          Boffin

          Re: "98.5% accuracy"

          Here we have a potential game-changer. It's potential is to capture asymptotic cases and direct them into the full test & trace system at no cost.

          Whilst it is an interesting result worthy of further investigation, it is not quite so simple. Consider the implications of "To that end, it is remarkable that our tool discriminates 100% of asymptomatics at the expense of a false positive rate of 16.8%."

          About 1 in 200 people in the UK have COVID. Hence, if you used a test like that across the whole population, then 70 million * 0.005 = 350,000 people with the illness (including the 40% (140,000) without symptoms) would be sent off for testing. However, a false positive rate of 16.8% would mean that 70 million * 0.995 * 0.168 = 12 million people (i.e. 17% of the UK population) who don't have COVID would also be sent of for testing.

          1. Cuddles Silver badge

            Re: "98.5% accuracy"

            While true, I think that's looking at it a bit backwards. The goal is to test the minimum number of people to ensure all cases are found. In the absence of any pre-testing, that means testing every single person. So a pre-test with a 17% false positive rate isn't bad because it makes you test 12 million people without an infection, it's good because it means you haven't had to test the other 58 million. Obviously it would be nice if it was even better, but as a first pass requiring viritually zero resources it's really very promising.

            1. Anonymous Coward
              Anonymous Coward

              Re: "98.5% accuracy"

              While true, I think that's looking at it a bit backwards. The goal is to test the minimum number of people to ensure all cases are found. In the absence of any pre-testing, that means testing every single person.

              I think it means testing every single person whom the test track and trace system has identified as potentially infected.

              1. Cuddles Silver badge

                Re: "98.5% accuracy"

                I think it means testing every single person whom the test track and trace system has identified as potentially infected.

                No. As I said, the goal is to find all cases. In the absence of any other information, that means testing everyone. A track and trace system is simply one way to try to filter the number of people who need testing. Early on it may have been pretty much the only option, but that doesn't mean it's the best, or the most effective use of limited resources.

        3. Steve K Silver badge

          Re: "98.5% accuracy"

          asymptotic cases

          ..one hopes that the infection curve does not progress this far...

    4. Anonymous Coward
      Anonymous Coward

      Re: "98.5% accuracy"

      > Sadly the journal is paywalled so I can't find out.

      What are you talking about? That journal is 100% open access.

      https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9208795

    5. jake Silver badge

      Re: "98.5% accuracy"

      All the justification needed is that people are throwing grant money at them, and possibly even start-up money, if they can figure out a way to add blockchain to the final product.

    6. jake Silver badge

      Re: "98.5% accuracy"

      Our DecalPercentage.

      Spot the reference, win a beer.

    7. FlamingDeath Silver badge

      Re: "98.5% accuracy"

      Shhhhhh, stop asking stupid questions, looks, it’s Ai, ooooooh shiney, so futuristic

      This dude, is selling a product, and you’re cramping their style

  2. Sorry that handle is already taken. Silver badge
    Headmaster

    "...from the way you expectorate"

    I'm not spitting into my phone.

    1. Neil Barnes Silver badge
      Coat

      Re: "...from the way you expectorate"

      But it's better to expectorate than never...

      1. pavel.petrman
        Coat

        Re: "...from the way you expectorate"

        Expectorate Patronum!

    2. jake Silver badge

      Re: "...from the way you expectorate"

      You "spit" into your phone every time you talk into it.

      Oh, wait, what am I saying ... talking on the phone is SO 20th Century.

    3. I ain't Spartacus Gold badge

      Re: "...from the way you expectorate"

      No-one expectorates the Spanish Inquisition!

  3. Danny 2 Silver badge

    Every silver lining has a cloud (of aerosol)

    I'd like to test that app, I've had a terrible cough since November. I mean it's been bad since 1985 but the last year it's been constant and painful. I know, I should see a doctor especially since I read that article by the young man who died of a brain tumour. Still, I don't want to go to the doctors during the pandemic in case it's nothing, and if it is something I'd rather not know. If it's still bad in a month then I'll cut back on my smoking. Meanwhile it's a useful way to get people to stand two metres away from me in shops.

    The uni's AutoID Lab earlier developed a speech-analyzing model to screen people for Alzheimer's testing, though turned their attention to COVID-19 as the pandemic took hold.
    That's actually true and obvious - my dad has dementia - but more so is the 'dementia shuffle', a characteristic walk without raising their feet. I suggested buying him 'wheelies' trainers but my mum opted for a wheeled zimmer frame.

    1. jake Silver badge

      Re: Every silver lining has a cloud (of aerosol)

      "I don't want to go to the doctors during the pandemic in case it's nothing"

      That's a piss-poor excuse at the best of times .... however, it doesn't answer the question of why you didn't go to the doctor sometime between November and March, before the pandemic became a known issue ...

      1. sgp

        Re: Every silver lining has a cloud (of aerosol)

        "If it's still bad in a month then I'll cut back on my smoking."

        Why not now?

        1. Danny 2 Silver badge

          Re: Every silver lining has a cloud (of aerosol)

          I was joking. If my cough gets worse then I'll have to cut back on the cigarettes but it won't be by choice. I'm twenty years older than I expected to be, those two decades have been okay but the next two I could pass on happily.

      2. Danny 2 Silver badge

        Re: Every silver lining has a cloud (of aerosol)

        Jake,

        I was convinced of the inevitably of a serious pandemic at the start of January, but I was also subject to other factors. The Scottish winter, the ill health of my parents, the fact to see a doctor I have to wait in line for two hours outside my health clinic, then sit inside for another two hours alongside undeniably sick folk often with coughs as bad as, or worse than, mine.

        Plus who do you think I have to excuse myself to? My grandpa taught me these lyrics to live and die for:

        I'll eat when I'm hungry, I'll drink when I'm dry,

        If the moonshine don't kill me then I'll live 'til I die

  4. chivo243 Silver badge
    Paris Hilton

    Medication

    My meds specifically mention stuffy and or runny nose, and dry cough as side effects, I have both now and then. Does this study\app take these conditions into consideration?

  5. Mathman

    Data set

    I wonder if the dataset included people who had other infections such as coughs colds and flu. Wouldn't be surprised if the input was people who tested positive and people who who have no symptoms. So it may be no better than saying have you got a persistent cough? Also those who know they have Covid may be recording under different conditions (physical and mental). So it would be important to test on a random set of subjects and only then test them for Covid with a follow up test a week later.

    1. Anonymous Coward
      Anonymous Coward

      Re: Data set

      Smokers/Non-smokers.

  6. Anonymous Coward
    Anonymous Coward

    Expectorate! Expectorate!

    Souhds like a Dominic Cummings-flavoured Dalek.

    1. pavel.petrman

      Re: Expectorate! Expectorate!

      - Why don't they just say spit?

      - Do you want them to save time?

      1. Anonymous Coward
        Anonymous Coward

        Re: Expectorate! Expectorate!

        Boris Johnson's Commemoration Stamps featuring Dominic Cummings will prove an abysmal failure.

  7. DS999 Silver badge

    This needs to be an app, not a "device"

    Put it into a device and we won't see it until the pandemic is over. Put it in an app and everyone can have it in weeks, and could test themselves if they are concerned if their cough is a problem.

    98.5% is FAR more effective than the temperature checks some places do before admitting people, it would be easy to have a (well protected) person stationed outside having people cough into a smartphone and let the app decide if they can enter or not. Sanitize the phone, repeat for the next person.

  8. cbars

    Useless

    Discerning a forced cough and COVID is as useful as asking, have you got a cough? But really, have you?

    Anyone with a cold/other cough in the sample...?

    1. 2+2=5 Silver badge
      Boffin

      Re: Useless

      Came here to make the same point about the forced cough but the survey is even more bollocks (or wondrous, depending on your pov) because the recordings weren't made when the person had Covid. So what the researchers have managed to produce* is an app that can distinguish between the forced cough of someone who tested positive for Covid an unknown numbers of days or weeks earlier, versus a forced cough from someone who tested negative for Covid an unknown number of days or weeks earlier.

      [*] According to the information in the article - I'm not going to spoil my indignation by actually reading the original to check!

      1. Mike 125

        Re: Useless

        >I'm not going to spoil my indignation by actually reading the original to check!

        Yea, I'm generally going with that too...

        "For asymptomatic subjects it achieves sensitivity of 100% with a specificity of 83.2%."

        'asymptomatic' adjective, definition: (of a condition or a person) producing or showing no symptoms.

        Anyone else see a problem here..?

        I call bullsh't, (but hope to be wrong.)

  9. LateAgain

    Not hard to check

    Use it at a testing centre and note the result on the samples sent off.

    It shouldn't take long to see how closely they match.

  10. Anonymous Coward
    Anonymous Coward

    pointless

    The whole exercise is pointless

    You are at your most infectious 24 hours *before* you show any symptoms at all.

    So quite apart from the large number of symptomless cases this won't do much to reduce transmission of the virus even if it works perfectly.

    The Loughborough/IMSPEX/Edinburgh/Dortmund breath test seems like a much more promising system.

  11. Luigi D'Goomba

    Phone placement is everything

    And if a male happens to place his phone on his nutsack and turn his head while he coughs, he can get a two-fer test for COVID and hernia.

    Thank you very much ladies and gentlemen, I'm here 'til Thursday.

    1. Steve K Silver badge
      Coat

      Re: Phone placement is everything

      and that's bollocks too.....

  12. CrackedNoggin Bronze badge

    MIT does fantastic work - but there is a lot of pressure and sometimes passionate optimistic fervor jumps the constraints - Cold Fusion, Food computers@Media Lab. I hope that is not the case here. I'm with holding judgement until the claim is properly validated by third party research.

    That it can attract invest money at this stage is actually not a good sign in my opinion.

    The people who would get hurt the most if this flops are the researchers involved.

  13. FrenchFries!

    Cystic Fibrosis

    Rather fascinating. I'm going to try to look at this objectively.

    My fiance' had Cystic Fibrosis. You can "generally" tell by by how they cough that they have CF.

    I know this is anecdotal and baseless from a data standpoint, but, the more we know, and the different approaches to knowing and understanding, hopefully help.

  14. Anonymous Coward
    Anonymous Coward

    Finall!y!

    A good use for wiretapping the entire population...

  15. Anonymous Coward
    Anonymous Coward

    Coming soon ?

    "Alexa. Covid check. <cough> <cough>"

  16. Robert Carnegie Silver badge

    Just thinking

    Who remembers Theranos pivoting...

    I didn't - I got it mixed up with "Theramed". Which is toothpaste and mouthwash apparently. Theranos was eyewash.

  17. Richard Pennington 1
    Coat

    The late, great Douglas Adams predicted ...

    ... telephone sanitisers.

    I'll get my towel.

    1. Robert Carnegie Silver badge

      Douglas Adams apparently did not invent the professional telephone sanitiser.

      http://www.guter.org/bristow/pretty.htm

      refers to a comicstrip character older than me, the lady from Kleenaphone who would visit the fictional monolithic office where "Bristow" worked. According to the article, readers never saw Miss Pretty, she came and went, but did encounter her colleague Miss Dimmitty.

      Douglas Adams's list of enemies of the people* also included hairdressers and we may be thinking twice about that now. I think the situation may have been that he didn't particularly care about the state of his hair, and he projected that onto hair being generally irrelevant to everybody.

      *The people of the planet Golgafrincham, and the enemies were sent off. Guess to where (did I explain this already?)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021