* Posts by Keshlam

5 publicly visible posts • joined 31 Jan 2022

Machine learning the hard way: IBM Watson's fatal misdiagnosis


Re: Much (most?) of Watson Health wasn't "Watson"

And "interesting" should be "interpreting" or some similar word. Sigh again.


Re: Much (most?) of Watson Health wasn't "Watson"

"reverb" was supposed to be 'even", of course. Darned auto-incorrect...


Re: Much (most?) of Watson Health wasn't "Watson"

Agile isn't the problem. Pretending you're doing agile without actually executing by those rules is. Mixing agile and waterfall leads to drowning in red tape.

You can have defined goals and do agile; you just need to be willing to change your path to those goals quickly, and to show (and demand) incremental progress en route. And, if necessary, to fail fast, accept that, and see if there's another good use for what you've invested so far or if it should be written off, shelved for possible later use, and the resources should be moved to a new goal.

I've seen good Agile, though we didn't give it that name, or any name, and maybe that's why it worked. Agile done properly reduces to "tell folks the direction you want to go in, encourage teamwork, help them set intermediate goals but otherwise get out of their way". Scrum and the other "here's how to do Agile" writeups can come close to that, but are already excessive formalism to reassure managers that they can still Manage and to give the bean-counters something to count And that's when they are executed as designed rather than letting the process become a drag in productivity.

Agile wasn't the problem. Not being agile enough might have been part of it.


I still tremendously respect the veterinary office...

... that named itself The Meow Clinic. Perfect.


Much (most?) of Watson Health wasn't "Watson"

Speaking as a Watson Health alum (admittedly a grumpy one):

While I agree with the analysis of why the "moon shot" health AI attempts failed, that isn't a complete explanation of why Watson Health failed. Watson itself was the flagship of that division, and (as others have noted) the branding, IBM quickly realized that much of not most of what the health industry needed was data standardization and sharing so that "big data" efforts --including any AI attempts -- could be applied at all.

Unfortunately, the company tried to apply the approach that worked when it was the Immense Blue Monolith: let a thousand flowers bloom, then prune back to whichever were successful. That works surprisingly well when you can afford to waste the work on "not quite" and duplication of effort, and are operating on basic research timescales. It's why IBM and Bell used to get so much strength from their research groups, after all.

But on Internet timescales, diving in without a plan doesn't produce results fast enough, especially when you start marketing before you actually have successful prototypes, or have reverb fully defined the question (as noted above). From inside, I saw a _lot_ of poorly defined goals, duplication of effort, and applying the wrong tools because someone in management had latched onto a concept as their salvation and didn't listen when the engineers told them it wasn't.

And in the process, IBM wasted enough time that the places where it could have taken the lead -- transcoding and standardization of data, for example, which as I noted was needed before much could be done with the data ocean -- were being addressed in other ways. Consider Datapower, for example, which IBM purchased. That was an effort originally started by the Mayo Clinic to address the problem of every doctor's office and hospital using different health record systems by interesting those, applying some patten matching to recognize what records referred to the same patient, and outputting a combined record with the patient's entire health history (or as much if it as those systems could provide). Hugely important when it was launched, for the health networks as well as big-data analysis. But as hospital networks absorbed each other and standardized upon a smaller number of record systems, the need for "live" analysis and synthesis started to fade, and with it some of Datapower's income streams... and meanwhile IBM was still flailing about trying to find the killer apps to run against that data, or even a good overarching architecture for them.

I believe it was an article in _Time_ magazine, some decades ago, which observed that IBM had transformed itself from "a battleship" to "a fleet of killer submarines" and had become a lot more nimble as a result. That was a more accurate comparison than the author realized -- it transformed the problem from one of concentrated inertia to one of distributed command and control. IBM still seems to be fighting to solve the latter, and in large part seems to be doing it by abandoning boats that get lost. But there's only so far you can get with that approach before it becomes a negative feedback loop and you can no longer afford to build new boats and expand your operations.

Especially when you start blind headcount actions so you're losing human capital and morale.

When IBM bought Red Hat, there was a lot of joking about "shouldn't Red Hat have bought IBM?" Looking at IBM's recent mission statements, it appears that this may in fact be the eventual outcome of that deal. Which might not be a bad thing for the industry, if it can happen fast enough that resulting combination (Blue Hat?) survives the transition.