back to article Top doctors slam Google for not backing up incredible claims of super-human cancer-spotting AI

Distinguished doctors have publicly criticized Google and others for making grand claims about AI research and then not sharing the source code and models to let others replicate and verify the experiments. In January, a team led by Google Brain's Scott Mayer McKinney published a paper in Nature boasting that its artificial …

  1. The Man Who Fell To Earth Silver badge
    FAIL

    Unicorns

    Publishing AI claims but not everything required for independent replication of the AI is like molecular biologists claiming to have squenced an organisms DNA but not publishing the sequence.

    1. RM Myers Silver badge

      Re: Unicorns

      It is more like not publishing the details concerning how the sequencing was done, how the organism was identified, how contamination was avoided, what reagents were used, etc. In other words, just publishing the results without the underlying data and methodology, which happens to be a very hot topic in biology and medicine currently. The whole Surgisphere fiasco provides a great example of what can happen when the underlying data (lying being the operative word) is missing in action.

      That is the reason there is a large effort to have research journals require that the supporting data and methodology be made available along with the published study, with appropriate controls to protect the privacy of study subjects. That way, other researchers will be able to verify that the reported results are consistent with the data, and have the information necessary to replicate the study.

      1. jason_derp Bronze badge

        Re: Unicorns

        So it's just classic corporate science then? "We have this cool thing, trust us, it's soooo cool. No you can't see, but it's super cool. Trust us. Changes everything. Nothing will ever be the same."

        [Product immediately ceases development once no profit is identified; never mentioned in public again.]

  2. HildyJ Silver badge
    WTF?

    Shocked

    "It's possible that Google might be holding back the code for commercial reasons."

    Ya think?

    I can fault Google for many things but trying to develop commercial medical software is not one of them. I doubt Stanford or MIT is any more forthcoming on their commercial products.

    Besides, there's nothing to prevent the boffins from developing their own cancer identification algorithms and if they do I assume that their respective institutions would patent and commercialize it.

    I'd welcome if Google or any tech giant developed a better breast cancer screening mechanism.

    Now if they started spamming the patients with reconstructive surgery ads, I'll happily put back on my Google bashing hat.

    1. bazza Silver badge

      Re: Shocked

      The problem is no one can verify if it really is a "better" screening mechanism. Google have said "it's better", and some data might temporarily suggest that is the case, but without a review there is no knowing whether or not there is a systemic problem undiscovered so far. Something like that can easily make it a "worse" screening system, which would harm patients.

      It's the same with the Covid19 tracking apps we're all supposed to be using. Neither Google nor Apple have actually said what they've done with the Bluetooth bit, so every nations' app built on top is having to trust that Apple and Google have got it right. For all we know it's rubbish, thus rendering the apps built on top of it worthless. In a global pandemic, that's not great.

      1. Joe W Silver badge

        Re: Shocked

        The main point for me was that "the web giant doesn't want its source code to be released until it's gone through a QA process"

        Then don't publish results on it if they cannot be trusted.

        No nature in Science and no science in Nature. They aim for hype and controversial articles.

        1. a_builder

          Re: Shocked

          Exactly

      2. Mage Silver badge
        Coat

        Re: Shocked

        At the time I expressed sceptism and said all these things are only pattern matching curated by experts.

        So even if it worked, you need to keep training the experts.

        So are Google trustworthy? Have they ever lied?

        IBM famously misled about Watson, which wasn't even the same Watson that won Jeopardy and didn't really work for Cancer diagnosis.

        AI is just hyped pattern matching using often biased and usually human curated data. If the so called "training data" (really a special kind of database) isn't expertly human curated confidence should be low.

    2. Schultz Silver badge
      Boffin

      "Google might be holding back the code for commercial reasons."

      The Google comment “It’s important to state that this is early stage research” does not indicate any imminent commercial application. So I'd consider it more probable that they sprinkled a generous helping of unicorn dust over their data and algorithm (maybe some p-hacking, aka tuning of the algorithm / data until the result conforms to expectations).

      Second law of scientific publishing: nobody is allowed to see the unicorn dust.

    3. CrackedNoggin

      Re: Shocked

      I agree that if Google develops commercial medical diagnostic software that is wonderful. Far better than earning 75% of their income from advertising. However, "Nature" shouldn't be an advertising vehicle. It should be a vehicle for open exchange of practical scientific information and always enable verification of experiments.

      They could set up a long term joint study with some cancer institute, running in parallel with existing diagnostics until those doctors became confident enough to incorporate the AI into their diagnoses.

      Then those doctors could publish their experience and result in a medical journal, and AI project could be expanded to other hospitals, without ever divulging the technical secrets. (Having someone else report your success would make a bigger impact.)

      Or they could have published in "Wired", who would be happy to hype the hype without supporting details.

  3. Imhotep Silver badge

    Just Give Google Everything

    The training material: were these from those health care records supplied to Google without the patient's notification and consent?

    Did that 'anonymized' set of data include date/time and location information?

    If so, can Google match a patient's data to an individual using cell phone and app location data?

    Is Google's end game that healthcare providers start providing Google with patient info for evaluation?

  4. EJ
    Trollface

    "We're calling the effort: Theranos II."

    1. a_builder
  5. Hstubbe

    If it is as good as youtube's content censoring AI, i would be very wary of this.

  6. Neil Barnes Silver badge

    claimed that most of the components in the model are open to the public already

    That may be so - but how they're joined together is what matters, along with the training data.

    In other news: this morning I have worked out how to travel faster than light, how to solve world peace, and how to play the flute. All the components of my method are open to the public already.

    1. Fruit and Nutcase Silver badge
      Pint

      Re: claimed that most of the components in the model are open to the public already

      I have worked out how to travel faster than light, how to solve world peace, and how to play the flute.

      So you were stumped when you attempted to solve the post Brexit frictionless border issue?

      -->a beer for you, if you do solve it

      1. bazza Silver badge

        Re: claimed that most of the components in the model are open to the public already

        Question though - a lager or a proper beer?

      2. Neil Barnes Silver badge

        Re: claimed that most of the components in the model are open to the public already

        I've solved it by emigrating...

        1. John Brown (no body) Silver badge

          Re: claimed that most of the components in the model are open to the public already

          There seems to be more and more of the wealthy Brexit proponents who heavily backed Brexit with cash and resources who, having achieved their aims, seem to be now leaving these shores to set up home in warmer climes, often tax havens.

          PS, no that downvote wasn't me. I don't know if you are Brexit supporter or wealthy :-)

          1. Neil Barnes Silver badge

            Re: claimed that most of the components in the model are open to the public already

            Neither. My plans were made long before the brexit fiasco crashed the currency, which has rendered my retirement potentially rather less comfortable than it might have been. Of course, if the UK rises to the challenge and the pound goes up, I'm better off - woohoo - but most of the more vocal pro-brexit industrialists seem to think that a low value pound is a good idea. Which it is, unless you want to spend it in, say, anywhere else in the world...

        2. Steve K Silver badge

          Re: claimed that most of the components in the model are open to the public already

          How does that help you play the flute?

          1. Cynic_999 Silver badge

            Re: claimed that most of the components in the model are open to the public already

            "

            How does that help you play the flute?

            "

            I think that was a euphamism ...

            https://www.urbandictionary.com/define.php?term=playing%20the%20flute

      3. Cynic_999 Silver badge

        Re: claimed that most of the components in the model are open to the public already

        "

        So you were stumped when you attempted to solve the post Brexit frictionless border issue?

        "

        I've already figured out a method to do that. I just need to solve a couple of minor technical issues. Such as how to time travel.

  7. Detective Emil
    Mushroom

    "vibrant ecosystem that supports future innovation"

    Not a response from the researchers, but from a PR drone.

    1. Tim99 Silver badge
      Unhappy

      Re: "vibrant ecosystem that supports future innovation"

      If you are a parasite, maybe you need good PR to keep many of your hosts?

  8. nematoad Silver badge
    Unhappy

    "...that is exactly why we scientists should always be transparent."

    Ah, but Google are not in it for the science. They are in it for the money. Yes they may have scientists working for them but the accountants and lawyers run the show.

  9. a_yank_lurker Silver badge

    Computer Code in Science

    The general problem when custom computer in research is the the quality of the code and the more importantly the quality of the algorithm. If either one is flawed then the ultimate conclusions are very likely flawed. When others can review both there is more confidence in then the conclusions. Admittedly, errors in the algorithm are often harder to spot and are likely to linger longer. Often now there is a tendency to treat computer code and algorithms as proprietary, intellectual property that is never published. It appears too many think they can commercialize the code and make a lot of money if they can keep it private.

    I remember a few years ago there was an economics paper (using an Excel spreadsheet) that had a typo in one of the calculations the authors missed. When it was pointed out the authors realized their mistake and pulled the paper. I believe the paper was peer reviewed so several people missed the error until after publication. The error was discovered, if I remember the details correctly, by someone reading the paper and digging into the spreadsheet and spotting the typo.

  10. Anonymous Coward
    Anonymous Coward

    dataset

    The problem is the training dataset, probably was used without any informed consent from the patients.

  11. a_builder

    Should never have been published.

    Certainly not in Nature!

    How was it peer reviewed without the source code? Otherwise this is hocus pocus science at its worst.

    Actually unbelievable that it was accepted never mind published. I’d never have got away with that and I’m first named author on 20 papers....

  12. FlippingGerman

    Weights

    Does anyone ever publish the structure of a neural network and the associated weights, saving everyone else the effort of training their models?

  13. Anonymous Coward
    Anonymous Coward

    google or nature?

    Who's to blame?

    The article makes out that the publication of the article without the source code is Google's fault.

    But I think Nature is equally responsible for accepting and publishing an article without the supporting reproducibility information. If they refused to publish, there wouldn't be a problem. Google does private research and keeps the results. Not an ideal world IMHO, but it is the existing one.

  14. Matthew Taylor

    Deflection from Google spokesman

    The Google spokesman said nothing, and promised nothing. Fancy that.

  15. RegGuy1 Silver badge

    Google -- don't be evil

    or is that gone?

    1. Richard 12 Silver badge

      Re: Google -- don't be evil

      Long gone. They removed that goal many years ago.

  16. Ptol

    Diagnosis as a service

    Surely if Google published a webservice that you could submit a sample and receive a non clinical assessment on that sample, then we could effectively run an extended trial of this service over time.

    That being google, one could still wonder whether the service software and model data remain constant over time, or were constantly being "improved"

  17. John Smith 19 Gold badge
    Unhappy

    Anyone recall "Iterated Systems" ?

    Claimed super-duper-astonishingly-good-you-won't-believe how good fractal compression.

    Had half a dozen pictures with massive compression ratios.

    Algorithm? Actually grad students stuck in office for months.

    Finally released software was basically brute force search for the coefficients and was sloooooow. Let's call Michael Beasley's claims (for legal reasons) "optimistic"

    This has a similar whiff. Missing detailed structure and critical parameters not listed. This should never have been given major attention in a scientific journal. Early publication credit for preliminary results fine. But where's the f**king follow up?

    My BS meter is redlining like a Geiger counter in the engine compartment of a Cold War era Soviet nuclear submarine.

  18. Mike 137 Silver badge

    Far from a unique problem

    This is just another example of a common outcome of the progressive commercialisation of scientific research over few decades. Almost all research is now grant funded or otherwise paid for by a party with a vested (typically financial) interest in the results. Consequently, complete openness (the essence of true science) is contrary to the sponsors' perceived advantage. For the same reason, it's also considered bad form to fail, so results tend to be hyped.

  19. RLWatkins

    Remember "quantum supremacy"...

    ... which Google claimed after they built a quantum-computer-like thing which basically changed state randomly, with no conceivable use, then claimed that it would take a conventional computer 10,000 years to simulate their device changing state randomly?

    They're a publicly traded company. Publicly traded companies create buzz to influence their stock price. Elon Musk is an expert at it, but they all do it.

    I'll believe the diagnostic AI when I see it do as well as a person which, claims to the contrary, it hasn't done yet.

  20. teebie

    Google Flu Trends

    Google also claimed they could predict where flu outbreaks would be, and they couldn't. Throwing the term 'AI' on top of the false claim doesn't make it any more true.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020