"All rather good points that ought to be raised or highlighted."
And answered.
Google is under pressure from a US lawmaker to explain how it trains and deploys its medical chatbot Med-PaLM 2 in hospitals. Writing to the internet giant today, Senator Mark Warner (D-VA) also urged the web titan to not put patients at risk in a rush to commercialize the technology. Med-PaLM 2 is based on Google's large …
"Med-PaLM 2 is not a chatbot; it is a fine-tuned version of our large language model PaLM 2"
Not as definitive of capacity as it sounds. According to Wikipedia "[a] language model is a probabilistic model of a natural language that can generate probabilities of a series of words, based on text corpora in one or multiple languages it was trained on". This is clearly not how the thought processes of a subject matter expert work, it's just a mechanism for generating statistically likely word sequences without reference to comprehension of meaning.
What's missing is conceptual understanding of the subject matter, and, although at best we can have it, that's something we still don't understand the mechanisms of, so we can't build it into any model. It's highly probable that we will never achieve that fully, as using the brain to examine the brain will always leave something unexplained because ultimately its an infinitely recursive process.
Essentially how this works is that it looks at the most common answers doctors give to a particular medical question, and prepares an answer based on that.
The problem is, the most common answer may not be the correct answer for this particular patient, and that seems to me to be a fundamental flaw of this technology.
> using the brain to examine the brain will...
Replicating how the brain works is getting into the wild and woolly realm of what is now referred to as "General AI" and even then only one view of it (another being that it doesn't have have to replicate how *we* do it...).
BUT that is NOT what we want from a medial system. We want something that can be verified and validated, something that can EXPLAIN how to reached a conclusion and how it rejected other paths.
We even train doctors to be able to explain their lines of reasoning, from the quizzing at rounds to the formal procedures of the regular mortality board enquiries.
> that's something we still don't understand the mechanisms of, so we can't build it into any model
We have had successes with explicitly creating models of reasoning chains, such as Expert Systems (XPs). Which are fundamentally capable of explaining how they reached a conclusion AND, far more importantly, how they have NOT been able to reach a conclusion, even including whether or not they have the ability to complete the task (but only if you can provide data on X and Y) or it is just outside their range. Unlike what we have seen from LLMs where they will gleefully reach any old conclusion, even if it is a pack of lies.
There are plenty of reasons why XPs and their ilk are not being used here (but that is a rant for when I'm not using a tablet's touch keyboard - ouchie). Suffice it to say that training LLMs is seen as easy (all you need is a shovel, CPU time and naivety) and there is no-one who has to stand up and take responsibility for the end result (there is no domain expert who wasn't as good as everyone thought, no Knowledge Engineer who clearly failed to translate his words into code).
Money + easy shovelling + zero personal responsibility = kerching!
"We have had successes with explicitly creating models of reasoning chains, such as Expert Systems (XPs)"
Yes, we have. But human mentation is strongly grounded in emotion, it's not just reasoning on the basis of logic. That's at least one of the bits we haven't been able to model, maybe because it needs a body as well as a brain as Bob Ornstein concluded several decades ago. I f he acted literally as described, Mr. Spock could not actually cope with the real world, and if you watch the first series attentively, you find that he does exhibit emotional responses all the time, albeit (because of his culture) with attempted restraint.
Are we concerned about creating a "general AI" or just a workable system for medical use, i.e. a workable system instead of the MediBot that Google is trying to foist on us?
Personally, right now, I'm only thinking of the latter and whether that can be usefully.
Philosophical questions about "can AI have emotions" are rather put of scope here, IMO.
PS
> he acted literally as described, Mr. Spock could not actually cope with the real world, and if you watch the first series attentively, you find that he does exhibit emotional responses all the time, albeit (because of his culture) with attempted restraint.
Vulcans were *always* described as being emotional - too much so, to the detriment of their society, so they practised restraint simply to stop themselves from constantly punching holes in walls (or anyone who happens to be around)[1]. All versions of Spock, TOS onwards, have been portrayed this way, as have other vulcans[2].
[1] beware of flying Plomeek soup during the Pon Farr.
[2] Spock's dad was an emotional mess during his last diplomatic mission, which was saved by dumping his mess onto Picard. Poor Jean-Luc.
> What is the frequency with which Google fully or partially re-trains Med-PaLM 2? Does Google ensure that licensees use only the most up-to-date model version?
Even more so than with other programs, with anything medical you shouldn't be asking for the latest version but the version with the least problems.
Especially when, as with any non-trivial neural net, you do not know what is actually going on inside and can not directly show that the newly-trained beast is actually better than the previous one: ChatGPT study suggests its LLMs are getting dumber at some tasks
This is a perfect example that computers are somehow not to be held responsible for what they do. Every software product you have specifically declares that its not responsible for any of its output, errors, malfunctions and whatever else. That's not good enough for medical devices, self driving cars and other safety critical areas.
People who build buildings get in trouble if the building falls down. People who administer drugs get in trouble if they do it wrong. So make the people that deploy these AIs legally responsible for its actions.
Yes, that would mean many fewer such things are deployed. Oh dear.