Machine learning the hard way: IBM Watson's fatal misdiagnosis It started in Jeopardy and ended in loss. IBM's flagship AI Watson Health has been sold to venture capitalists for an undisclosed sum thought to be around a billion dollars, or a quarter of what the division cost IBM in acquisitions alone since it was spun off in 2015. Not the first nor the last massively expensive tech biz …
The main thing shared by Jeopardy Watson and medical Watson was branding. Winning at Jeopardy is a parlour trick anyway.
Current AI is really pattern matching. So for medicine you need a vast database of human curated data, by experts in each field. Even if it worked it might eventually be self defeating as eventually there might not be the human experts to diagnose new, unrecognised data.
>Current AI is really pattern matching
This really isn't true in the sense you're implying it. Modern ML techniques are more than capable of recognising previously unseen patterns and equally of generating novel patterns unlike those seen before. They're also capable of learning in an unsupervised manner, though this would probably be ill-advised in this context.
Bizarre that this accrues so many downvotes when it is such a straightforward statement of fact. Am I missing something? Describing "current AI" as "pattern matching" is really very far from the truth - even if one is generous and assumes the OP meant "pattern recognition". Likewise as is the description of hard dependence on labelled training data and most particularly the idea that a model is constrained by what it has seen before, thus risking stagnation or myopia.
Practical examples of models detecting new, previously-undescribed-by-human patterns are increasingly common. In particular drug discovery as a field is increasingly dominated by ML models (which will be branded "AI" because of PR people) doing exactly that - discovering previously unknown patterns following techniques we can't really explain.
In fact the societal burden is going to be the complete opposite of what the OP implies. It's not going to be a risk of trained models failing to innovate - it's a current & real problem that models are already innovating in ways we can't explain, and therefore struggle to regulate.
Finding a new pattern in a large dataset does not mean you aren't pattern-matching. It just means you had to find the pattern as well as all the matches. Since guessing patterns and then trying them out is an embarrassingly parallel problem, you'll have to forgive us if we are sometimes underwhelmed by AI.
Being able to discover new patterns to recognize - defining new techniques for recognition and new entities to recognize - is very much doing more than just pattern recognition. It's conceptually the difference between being able to write (and verify!) a set of instructions to be executed and simply being able to execute a set predefined.
You could argue that new pattern synthesis is in fact just a form of pattern recognition in its own right, but you're into turtles-all-the-way-down territory there, especially given as a field ML is already well into the domain of models-building-models territory with transfer learning.
Which is a big part of why we're already off the deep end in terms of being unable to explain the behaviour of many common models - they were in turn built from (and often by) other models. We've got NFI how they work internally. We can test the results, less so the structure.
Reduction of ML as a field to "guessing patterns and then trying them out" is a gross oversimplification. That is very far off the mark for how all practical ML techniques work.
Actually, in my limited view it could be doing LESS than just pattern recognition.
At least with pattern matching, you can take the pattern and discern why something did/did not match.
With AI/ML, it may discover the 'pattern', but from what I understand of the current state it cannot explain why.
Don't get me wrong - I think it can be very useful as you mentioned. But I think that until it has explanatory power for why it is in a way less than pattern matching.
You're absolutely right, but this is a subtle and contextual thing. For one there are many areas where we actually can explain what an ML model is doing and why. So-called "explainable" ML (or XAI for short - XML was taken) is a hot topic of research because in many areas it is a hard requirement that we be able to say why a decision was taken.
This is particularly relevant in areas that directly impact people's lives. Say, for example, why a loan was approved, why a fraud investigation was triggered or why a clinical care decision was taken. Explainability is a hard requirement there, and that requirement is built into legal frameworks like GDPR.
But in others it matters a lot less. I'll look at drug discovery again. Do we really care how a model developed its understanding of which drugs "might" work? We already have robust tests for understanding which drugs do work, and we have a long and illustrious history of using drugs without a full understanding of *how* they work. So does it ultimately matter when the model is able to apply itself hundreds of thousands of times faster than any human?
The answer is still yes, but it's a very different kind of explainability that's needed, more concerned with things like informing subsequent manufacturing pathways and outcome prioritization than justifying a decision.
But hey, shouldn't be that hard to explain right, after all it's just a bunch of pattern matching and brute force, or so el reg's enlightened commentards have so assuredly told us!
"So does it ultimately matter when the model is able to apply itself hundreds of thousands of times faster than any human?
The answer is still yes, but it's a very different kind of explainability that's needed, more concerned with things like informing subsequent manufacturing pathways and outcome prioritization than justifying a decision."
Yes, it does ultimately matter that the knowledge has a why. That is because knowledge should not siloed. That is, knowledge of A in field B leads to discovery of X in field Y.
Without understanding why, you have created yet another knowledge silo, but one that nobody (at least human) has visibility into.
Being a medic, I read El Reg as a personal hobby. And yes, I have read about medical IA and so on professionally, but no, I don't understand its inner workings. Yes, tech really, really enhanced our abilities to treat. But...
Everybody in health care will tell you that, in the end, the pivot of medical treatment is an interaction between people.
Knowing this, I want to put this to you techies here:
How would you personally like to be treated for your (potentially life impacting) ailment by IA exclusively? Never see a person you can talk to and interact with? Just like all those other services who give you a "convenient selection module", "Please press hash to continue"? Just imagine you yourself stuck in that phone booth like medical pod with your cancer... I apologise for the dramatic imaginary for demonstrative purposes, but we're talking your life here. Or you having pain every day of your life, not you complaining your app is not working as advertised (Ever tried to talk to a real person there with your customer complaint?)
I'm not naive, and I realise that there are always "great opportunities" out there to "exploit and monetise". But maybe we also have to think at a certain time when we do something, whether we really need it (that way).
-- How would you personally like to be treated for your (potentially life impacting) ailment by IA exclusively? Never see a person you can talk to and interact with? --
Don't know which country (or planet) you've been in the last two years but that's almost what's happened with the NHS in the UK. Get to see an actual doctor or nurse - nope - have a phone appointment with an anonymous voice or better yet use 111 and talk to someone reading from a script (you might be lucky after playing 20 questions to speak to a medical professional).
I have to strongly endorse this view: for historical reasons, I have been trying to have an in-depth conversation with my GP for more than a year - but I have found it impossible to contact her. The GP practice phone number has a recorded message which directs me to use an on-line system. That system does not allow me to send a message to my GP. I have taken to hand-delivering letters to a letter box in the wall of my GP practice building - I cannot get in the building because the door is never unlocked. 3 letters so far have produced no result. I think that my next step is to take legal advice. What a sorry, ineffectual state the NHS is now in.
Don't know which country (or planet) you've been in the last two years (LybsterRoy)
I have to strongly endorse this view... (A.A.Hamilton)
I'm very sorry to hear this. And to answer the question directly: the last two years I have not been in the UK, partly driven by the exact reasons you mention, but then experienced "from the other side".
Unfortunately I must also admit by now that it is certainly different elsewhere. Or perhaps more fair to say, can be, regarding the grass in all the geographic experiences I have been able to experience.
Sorry to answer so late. My father worked in a Glasgow University research unit in the 70s which was doing what we would now call deep learning in medicine. They used a PDP 11 (!!!). The unit studied gastrointestinal diseases. They built an easy to use terminal with yes/no style buttons as people then would have been scared off by a keyboard. They found that people answered a computer more truthfully about embarrassing GI symptoms.
OF COURSE the computer system was never used to deliver a diagnosis to the patient.
https://www.researchgate.net/profile/Robin-Knill-Jones/publication/3592410_Evaluation_of_a_statistical_diagnostic_system_GLADYS/links/56a2752d08aef91c8c0ef61f/Evaluation-of-a-statistical-diagnostic-system-GLADYS.pdf
Humans see patterns in all kinds of things where none exists. We gaze at clouds and see shapes; we look at the stars and make tales of gods, heroes and heroines; we hear that someone's aunt died a week after being vaccinated and conclude that vaccines are dangerous.
From machines, we expect explainable fallibility.
From my admiittedly limited understanding of the subject, it's possible for it to both use brute force pattern recognition and for that to yield results that are not easily explainable to humans who can't grasp the enormous datasets they are based on in their heads.
"Do we really care how a model developed its understanding of which drugs "might" work? We already have robust tests for understanding which drugs do work, and we have a long and illustrious history of using drugs without a full understanding of *how* they work. So does it ultimately matter when the model is able to apply itself hundreds of thousands of times faster than any human?"
Yes, of course we care. We have a long history of using drugs without understanding how they work. We also have a long history of royally screwing up because of that, with a huge variety of examples that turned out not to work as thought, or even actively making things worse. One of the biggest things distinguishing modern medicine from the vast majority of that long history is that these days we make the effort to actually understand how and why things work, and to use that knowledge to improve things further.
Your last quoted sentence is probably the important part, and is precisely why so many people here are down on AI. If a computer model is able to blindly test things faster than humans could do it themselves, that's great. But that's nothing new; it's what computers have been doing for the better part of a century now, and is exactly what you would expect from them. It's not AI doing something amazing and new that brings a real difference in kind to the way things are done, it's just the same blind testing of lots of different things to find out which ones work. There's nothing wrong with that if it helps do the job faster, just don't pretend it's anything more than that.
I'm not curation is really required. What you do need is a more extensive corpus of data so that co-factors can be spotted more easily. Simple statistical analysis should indicate areas worth investigation and testing.Unfortunately, medical staff have neither time, skills or equipment to do the work. And, the industry doesn't help by wanting to keep all the data for itself for future commercial products and services.
And Watson messed that up too. It never warned that "bad salad dressing" could be the cause for my belly pain. Another fail for Watson. It can deduce questions from answers, but it cannot diagnose and apparently has a terrible track record at deducing bad food consequence. Is there anything you can use Watson for if you want answers?
IBM Watson presents solutions in business automation, adtech, financial services operations, customer service process automation, video streaming and presumably the proverbial kitchen sink. "Watson" is a grab-bag of completely disparate, un-like, packaged technologies mostly taken from IBM's legacy base or bargain-basement acquisitions and lumped together under a common brand based entirely off a minor PR coup achieved by a tech demo on TV ten+ years ago. These tools have nothing in common with each other.
Several years ago when I was still working for IBM, and Watson was just beginning to show signs of stalling out, I told a friend and colleague: "You'll know that Watson is failing if IBM starts tacking the brand onto unrelated products in order to goose the revenue and sales claims."
Within a year it was calling all sorts of entirely conventional analytics "Watson". And soon after, it was buying perfectly ordinary healthcare products and calling those Watson Health.
And here we are in the inevitable endgame.
He also smoked a short clay, a briar or most often a Churchwarden, not a Calabash. The Calabash was selected by Basil Rathbone I believe as it covered a smaller area of his face than the other types and allowed him a greater range of facial expression as well as being easier to photograph in atmospheric lower lighting conditions as it sits in the same focal plane as the face.
Apple had Sherlock, a universal search engine that got decent results and grew to become Spotlight and Siri. IBM built Watson, thinking they would be able to analyse, not just search, any data. But just like in the stories, Watson remained clueless while Sherlock kept finding crucial information.
At least Watson was an appropriate name, in all the books / stories, Watson never really masters the Holmes skills of deduction / problem solving
Except that he was a pretty good doctor.
Do not judge the character by the buffoonery in the old B&W films. He wasn't Sherlock's equal in criminal investigation but Holmes did not suffer fools and wouldn't have shared an apartment with him unless he was an intelligent and capable man.
He was also an Afghan war veteran, from the second time the UK Government and British Army thought it was a good idea to try to fight there. He got wounded at Maiwand, which is fairly close to Lashkargah nowadays, there's a highway that runs from Kandahar to Lashkargah that passes right through it. One of the many highways and roads in that country that I've been shot at on myself.
Watson was no fool, he was tough as nails, intelligent, a good shot, flexible and mentally agile enough that he could put up with weirdness out of Holmes, and was a very good physician
"Watson was no fool, he was tough as nails, intelligent, a good shot, flexible and mentally agile enough that he could put up with weirdness out of Holmes, and was a very good physician"
Good enough that Holmes had to be sure to have Watson keep his distance from him when feigning illness (in The Adventure of the Dying Detective)...or Watson would've caught on and spoiled the attempt to nail a real poisoner.
As far as I understand it (thirty seconds on Google), Jeopardy gives you an answer and you have to respond with the correct question. This likely isn't that hard to do if you have a fast processor and access to a massive data set.
Medical diagnosis, on the other hand, is a much more subtle affair. You need to listen to what the patient is describing, and spot not only the important parts of what is being said, but also what is not being said. Using this, to then ask the right questions to narrow down possible conditions without leading the patient. There are many reasons why somebody might wake up and puke over the floor, from pregnancy to poisoning. It takes at least a decade of training to know what to look for, and a lifetime to know how to look for it.
I don't hold out any hope that AI in its current form is something that should be let anywhere near an ill patient.
Jeopardy gives you an answer and you have to respond with the correct question.
I am dying of curiosity: has 42 ever been dealt on Jeopardy and what was the question? I tried to google, but the only vaguely relevant thing on the 1st page of results was about someone named Amy Schneider who apparently was a Jeopardy winner at some point and, possibly independently, was 42 years of age at some point. Doesn't quite cut it, IMHO...
The size of the data set wold be constrained by the column the question was in, which would make it easier to hone in on the probably answer.
Say the column is "science fiction" and the answer is "42", you're suddenly dealing with a far smaller set of possible answers
@T. F. M. Reader
Don't go down the Jeopardy / Amy Schneider route...
Amy Schneider is a trans woman & one of the top ever Jeopardy TV series players, so you will soon end up on web pages referencing Amy Schneider that are basically full of arguments / insults from people on different sides of the "a trans woman is a woman" argument people lie, and like most trans related "discussions" they usually degenerate into toxic unpleasantness.
"Douglas Adams was a programmer, and knew that 42 represented the *, which can mean anything. So the answer to the question of "life, the universe, and everything", means anything you want it to be."
Every day is a School Day - thank you for pointing out that crucial piece of information.
"As far as I understand it (thirty seconds on Google), Jeopardy gives you an answer and you have to respond with the correct question. This likely isn't that hard to do if you have a fast processor and access to a massive data set."
Not to be rude, but you really don't understand it very well. Jeopardy questions/answers are very hard to figure out, often more like crossword clues than pub trivia, and the best humans are highly regarded for both their knowledge and skill. And Watson, like the best human players, was also really good at introspecting on how confident it was of its answer, and therefore whether to take the risk of buzzing in (Jeopardy penalizes for wrong answers).
It really did take a lot of work to get Watson to the point where it could not only win the game but also avoid making a fool of itself (early iterations were sometimes hilariously bad).
The problem with Watson's Jeopardy win is that Watson achieved something very difficult... that did not translate into anything financially valuable in the real world. It solved a problem that nobody had.
but also what is not being said.
Absolutely.
I'm not a doctor. My professional diagnoses were purely limited to understanding and resolving blocks in kids' learning of basic skills and to some extent behaviour issues.
I'd have achieved a lot less if I'd just tried to work with what I was told, rather than trying to spot what I was not being told. And much of that was tangential to the reason for referral.
Some very simple examples from my earliest days.
I was told that an eleven year old girl was spelling randomly. Her reading was fine and her writing in general wasn't terrible.I looked at her spelling and it did seem very strange. But it didn't seem random. There was some sort of pattern within it. No one had mentioned that she was Portuguese - but she had an accent so I asked. And then I checked some examples of Portuguese speech patterns. And then went back to the writing samples. And of course you can guess the rest. I confirmed it with an adult native speaker of course.
In a number of other such cases - "random spelling" and the good old "they write their letters back to front, must be dyslexia" even though the kids weren't having any other problems with literacy- a quick look at how they formed their letters and a referral to the OT service resolved the issue, almost magically.
Diagnosis is so often about what isn't being said.
Exactly this, plus another important thing.
Watson and most other medical ML has a core fantasy that most patients come in and *list and describe their symptoms verbally*. This is total nonsense.
Firstly, an experienced GP has a fair idea of where the conversation is going to go just by watching the patient come into the room, before they’ve said anything. How they walk, posture, complexion etc. When the three year old is just very quiet and floppy.
Secondly, most patients, put bluntly, are fairly much the opposite of articulate college-educated students and professionals, and at difficult times of their life. People with dementia, depression, or shall we say non-standard health beliefs. And they hide stuff. How helpful will Watson be to someone with dementia just telling them that? Maybe the patient has insight, maybe they don’t, but just diagnosing them with some IQ test and giving them a score would count as the least useful “doctor” on the face of the planet.
Thirdly, it’s notorious that people tell you the important stuff while they have their hand on the doorknob to leave. After they’ve told you their arthritis is painful, and the blood-pressure lowering drugs don’t agree with them, if your consulting skills are very good they might just tell you on the way out that they aren’t sleeping so well. And if you are alert you find out that they are suddenly getting drenching night-sweats. But they think it’s because they are gluten-intolerant. Which when you enquire further, they’ve decided because they are suddenly having difficulty going to the loo.
Where is Watson in all this?
Hardly needs a genius to diagnose that *after they’ve winkled out the history*.
I have been dealing with various groups for several years, all asking for the same things 'what are the Key Performance Index values for X, Y, Z?'
It started with a fresh batch of Data Scientists - with shiny new degrees, who had never seen a production workload. Who completely failed to understand that the performance metrics for XXXX might be entirely different from YYYY and that it is not a problem because they are running different types of work. Or that there are fields in the data being examined which are there for historical purposes but mean nothing today. "We spotted an anomaly in the chip bin full switch!" It bothered me to no end when I was told (in that lofty tone) that a Data Scientist does not have to actually know anything about the data being evaluated. Then when that project fizzled out one of the reasons given was 'they never understood the data.'
There are some rules that weary people have been programming into various monitoring tools for decades, and I have provided many lists over time of what is 'always true' (short on storage is always a problem), what should be monitored/watched for trending behavior, what MAY be an issue, and what should be ignored. I continually get asked to review results that highlight the new pattern in the chip bin full indicator, which is caused by the lazy person who originally took out that bit of code failing to initialize that halfword.
My sister has been heavily involved in looking at ML approaches to analysing images for cancer. She's a doctor who has been doing this stuff in person for years. When she got involved with the ML people she said - it's absolutely critiical that the people working on the algorithms etc have an idea about what the data really means.
The worst bit is when they train their algorithms on the imaging presentation data instead of the raw data. Every manufacturer applies their secret sauce to the raw data before it's shown to a physician. Trying to highlight important structures while keeping out the noise. This means an algorithm trained on one manufacturer will only work for that manufacturer AND only at that time. As the manufacturers adjust these post processing things all the time.
You can get the raw data from the machines but that generally requires extra setup steps and those images are generally not stored in PACS as they're so much larger.
Whenever I see a study or press release about some magical new AI/ML algorithm and they say they trained on the presentation images I basically ignore it as the algorithm is useless in the real world.
I am but a lowly programmer, however, when a customer asks me to export data from a data store, I find it incredibly important to know what that data is in order to be sure that I export it correctly.
And if the customer is asking me to import data, then it's time to sit down and map every single piece of data that is supposed to come in and where it's supposed to go.
If you're working with data and you think that you don't need to know what it represents, you need to change jobs.
It's that it died a perfectly predictably death under the idiotic top heavy process driven we have more lawyers and accountants than scientists ethos of IBM. They got rid of most R&D long ago and decided to purchase all their expertise. The problem was in then forcing all those bright young acquisitions into the Big Blue Blockhead Method. Every acquisition chained the top talent to the firm for 2-3 years, they vest and then run for the doors. Why? Because no small young aggressive nimble company wants to work for 13 vice presidents who all want status reports and detailed metrics on a daily basis.
Speaking as a Watson Health alum (admittedly a grumpy one):
While I agree with the analysis of why the "moon shot" health AI attempts failed, that isn't a complete explanation of why Watson Health failed. Watson itself was the flagship of that division, and (as others have noted) the branding, IBM quickly realized that much of not most of what the health industry needed was data standardization and sharing so that "big data" efforts --including any AI attempts -- could be applied at all.
Unfortunately, the company tried to apply the approach that worked when it was the Immense Blue Monolith: let a thousand flowers bloom, then prune back to whichever were successful. That works surprisingly well when you can afford to waste the work on "not quite" and duplication of effort, and are operating on basic research timescales. It's why IBM and Bell used to get so much strength from their research groups, after all.
But on Internet timescales, diving in without a plan doesn't produce results fast enough, especially when you start marketing before you actually have successful prototypes, or have reverb fully defined the question (as noted above). From inside, I saw a _lot_ of poorly defined goals, duplication of effort, and applying the wrong tools because someone in management had latched onto a concept as their salvation and didn't listen when the engineers told them it wasn't.
And in the process, IBM wasted enough time that the places where it could have taken the lead -- transcoding and standardization of data, for example, which as I noted was needed before much could be done with the data ocean -- were being addressed in other ways. Consider Datapower, for example, which IBM purchased. That was an effort originally started by the Mayo Clinic to address the problem of every doctor's office and hospital using different health record systems by interesting those, applying some patten matching to recognize what records referred to the same patient, and outputting a combined record with the patient's entire health history (or as much if it as those systems could provide). Hugely important when it was launched, for the health networks as well as big-data analysis. But as hospital networks absorbed each other and standardized upon a smaller number of record systems, the need for "live" analysis and synthesis started to fade, and with it some of Datapower's income streams... and meanwhile IBM was still flailing about trying to find the killer apps to run against that data, or even a good overarching architecture for them.
I believe it was an article in _Time_ magazine, some decades ago, which observed that IBM had transformed itself from "a battleship" to "a fleet of killer submarines" and had become a lot more nimble as a result. That was a more accurate comparison than the author realized -- it transformed the problem from one of concentrated inertia to one of distributed command and control. IBM still seems to be fighting to solve the latter, and in large part seems to be doing it by abandoning boats that get lost. But there's only so far you can get with that approach before it becomes a negative feedback loop and you can no longer afford to build new boats and expand your operations.
Especially when you start blind headcount actions so you're losing human capital and morale.
When IBM bought Red Hat, there was a lot of joking about "shouldn't Red Hat have bought IBM?" Looking at IBM's recent mission statements, it appears that this may in fact be the eventual outcome of that deal. Which might not be a bad thing for the industry, if it can happen fast enough that resulting combination (Blue Hat?) survives the transition.
Quoth Keshlam: "... on Internet timescales, diving in without a plan doesn't produce results fast enough, especially when you start marketing before you actually have successful prototypes, or have reverb fully defined the question (as noted above). From inside, I saw a _lot_ of poorly defined goals, duplication of effort, and applying the wrong tools because someone in management had latched onto a concept as their salvation and didn't listen when the engineers told them it wasn't."
That's what it comes to when everyone wants to go Agile.
Agile isn't the problem. Pretending you're doing agile without actually executing by those rules is. Mixing agile and waterfall leads to drowning in red tape.
You can have defined goals and do agile; you just need to be willing to change your path to those goals quickly, and to show (and demand) incremental progress en route. And, if necessary, to fail fast, accept that, and see if there's another good use for what you've invested so far or if it should be written off, shelved for possible later use, and the resources should be moved to a new goal.
I've seen good Agile, though we didn't give it that name, or any name, and maybe that's why it worked. Agile done properly reduces to "tell folks the direction you want to go in, encourage teamwork, help them set intermediate goals but otherwise get out of their way". Scrum and the other "here's how to do Agile" writeups can come close to that, but are already excessive formalism to reassure managers that they can still Manage and to give the bean-counters something to count And that's when they are executed as designed rather than letting the process become a drag in productivity.
Agile wasn't the problem. Not being agile enough might have been part of it.
Even actual doctors trained for ten (oooh) years aren't that great. Autopsies, which the medical profession is glad to see phased out, typically show that around 25-30% of patients are misdiagnosed. See https://www.newscientist.com/article/dn27733-death-of-the-autopsy-leaves-us-in-the-dark-about-misdiagnosis/
True, except for the snide remark "glad to see phased out".
No Doctor I have ever known has been glad to see autopsies phased out.
They are phased out by management (who don't want to pay) and insurance companies (who benefit from medical failures*) and by relatives, who aren't interested: she's 97, was deaf and blind, and now you want to cut up her body because the /doctors/ want to cut up bodies?
*every medical failure emphasizes the importance of having insurance. Floods sell flood insurance: fires sell fire insurance.
There was an excellent episode of Think with Pinker about the knowledge work of medical diagnosis
https://www.bbc.co.uk/programmes/m001283l
In it was described the process of discovery, recalibration and new pattern seeking that's involved. It's called Bayesian updating. Did the forget to put that into Watson?
Watson never qualified as actual Artificial Intelligence (AI). It qualified as an Expert System with speech-to-text and text-to-speech, IBM style, grafted on. Research into Expert Systems has been going on since the 1950s. They never were properly considered to be AI. Only marketing considered it to be otherwise. Watson over-promised and under-delivered. Within its niche, its was a brilliant accomplishment. But to hand it over to MDs as a diagnosis and treatment partner was unrealistic, imaginary, the victim of hype. Most "AI" of our current day is hype.
Before real AI makes a real mark and provides a real benefit to we humans, we have to dump the spin and begin to be realistic about what actual "AI" can do. It can't actually think beyond taking orders, interpreting orders, scanning a database, interpreting the data, then handing over the best result it could find within its interpretive limitations. Those limitations are vast and are going to remain so for the foreseeable future.
Consider the fact that we humans are ourselves vastly limited. We take in perceptions that are consistently faulty and incomplete. We conjure our own interpretations of the faulty data and push them through filters that vary according to our personality, experience and other Inner World influences. As I put it, we never know everything about anything. Take a look at the huge cost of malpractice insurance and you'll get an idea.
We can create machines that can compensate for human perceptual failings and calculations. But considering the limits on our own 'intelligence', it's nonsense to expect any AI to do any better than we can at coming up with correct answers to problems. What AI is good for is the providing of another perspective that can be immensely useful in addition to our own interpretations, output and outcomes.
AI is only a tool, as all computing is only a tool. If the tool doesn't fit the job requirements, drop it and find a better one. AI won't prove better than communicating with a fellow human who has different, if not better knowledge and advice than you do. Thinking is the best way to travel. Pretending a machine tool can do more than it's realistically designed to do is not thinking.
Nope. Most physicians are under absolutely disgusting workloads so would love to have a tool that can lighten that load. However it has to basically work out of the box as they do not have the time to baby the system or completely relearn the entire workflow overnight.
Yes there are some fears from some generally older physicians and that's understandable. But the current and near future systems are so limited that it won't happen. And even long term you'll still need someone that can translate the output of these systems into something that the patients understand and accept.
Just as code automation and high level languages have not replaced the need for software developers. AI/ML SaMD will not replace physicians.
In the company I work in Watson is generally hated for making our lives needlessly difficult.
AI/ML in medical software has a role and if done properly is amazing to see as it performs as good or even better than humans without getting tired. BUT, it is no panacea and has very important restrictions to keep in mind at all times. As long as you stay within those restrictions it's brilliant.
Also there are basically no serious companies that have products out right now that advertise as being diagnostic. Diagnosis is disgustingly complex even before you start thinking about liability. The company I work for makes it clear as part of our regulatory approval that we are only advisory. We can help highlight pieces to information that are likely to be useful. But the physician must use their training, experience, and knowledge of the patient and their situation/culture to make the final set of judgements.
The biggest area that Software as a Medical Device (SaMD) is going to be useful in the short to medium term is to help identify interesting areas or data for physicians. Being a second pair of eyes to help catch stuff they may have missed due to fatigue. Meaning they can spend more time on the hard stuff and get more people the care they need. Especially in the sub specialties.
Since starting my current role I've learnt a shocking amount about medical software. When done properly it truly is world changing. Watson was not done properly.
"But the physician must use their training, experience, and knowledge of the patient and their situation/culture to make the final set of judgements."
The last time I relied on this they decided without any discussion that it was an age related problem. Job done.
I deduced through intermittent use that the problem was extended use of a soft bicycle saddle. An age related decision as in my bike downgrade I had forgotten why the best saddles are hard.
So I'm inclined to take the original statement with a pinch of salt.
I'm not saying physicians are perfect. Far from it. In general they have a better understanding of the patient's culture and situation than an algorithm looking at raw data.
As much as it sucks it's important to find a physician that you work well with. I know it's not often possible for people to try out physicians until they find the one they click with.
An interesting lesson I learnt from my work us about a measurement in mammography that's useful to help identify women who will benefit from extra screening. This measurement involves placing the woman into one of four groups based on breast density. The problem is currently the measurement is done visually and thus by the individual judgement of the radiologist. This means any two radiologists will only ever agree on the categorization of breast density about 60-70 percent of the time. And even the same radiologist will disagree with themselves if they categorize the same image a couple years apart. With modern image processing we can build a repeatable and physics based measure of breast density. But you'll still get physicians that disagree with the measure as they missed some important part like the thickness of the measured breast. SaMD is a really interesting field.
"ironic, since the game of Jeopardy at which it excelled is all about deducing questions from data"
Nah, Jeopardy is all about answering questions, but with things stupidly and tortuously phrased to fit into a gimmick which stopped being cute about forty years ago. Good lord, I wish they'd give up on it already.
I'm a blatant IBM hater for many decades. No idea about Watson, thought it another IBM BS move. Last year took on a Fortune 50 client that has a division who swears by Watson. My team discovered after not a lot of research that their performance was down about 35% less than the rest of the company who wasn’t using Watson on the specific business process they were cheerleading it for.
Irony times three. They approached me late last week to inquire as to whether we would like to take over maintenance of the former IBM contract. I informed them that we presented to the C-level and our main goal is to rip and replace. Their gnashing of teeth made my single malt go down much smoother Friday evening.
I saw a demo of Watson a couple years ago at an IBM sales pitch. The idea was that Watson had been fed a guidebook about the building, and you could ask simple questions like when was the building built? Who lived there?
Of course even this simple demo failed utterly miserably. The technology does do something I’m sure, but dammed if I saw it that day.
> A good doctor sees the patient, not the symptoms. Watson saw the symptoms of inefficiency and lack of capability. It did not see the process of care and making whole, where doctors, not data, were what needed to be understood.
This is probably true, but I think not often reality. I don't mean not all doctors are good (although that's true as well) but because of specialisation, say, a radiologist won't take everything of a patient into account when assessing them. Neither will a cardiologist, or an oncologist. And how could they? The quoted definition of a good doctor is absurdly hard with every year of more specialised knowledge being discovered about every aspect of medicine. Watson if anything could help more with general diagnosis that must reach across unusual areas of medicine than with detailed, non-whole-patient analysis.
I posted this here after another Watson Health story and got flamed, but I still believe there is at least -some- truth to it:
"Isn't it possible that [at least some of] the cause of Watson's failure in medicine is that the people charged with making it smart and good were the very people that it would replace?"
Also, that people with 10 years of medical training don't necessarily know jack about computers?
I'm slightly concerned that diagnosis can't be automated. How is a doctor's intuition squared with the scientific method?
Surely the implication is that as human doctors skills must vary enormously, diagnosis is basically random?
Or was this failure about things other than diagnosis?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
