Re: All this proves is that ...
> The tech behind LLMs was inspired by the brain's neural network.
Inspired by how we thought we might sort-of simulate neurons using simple techniques. The application of weightings was (is) a simplification of what was believed to be how neurons function back in the 1950s. Our understanding of how neurons mechanistically function has changed since then (apparently, it is a bit more complicated than multiplying big matrices of weights, involving icky chemicals that can be modified by all sorts of other chemicals in a soup bowl) and most of what has been done with the computers is to make them bigger and bigger (and reduce precision of the numbers used in the models, not because we've learnt that is how Nature does...).
In other words, unless you have some very good citations to back it up, don't go around thinking that what is going on inside an LLM is in any way related to what Gordon inside our heads - and most certainly not comparing it to *all* that goes on inside their (we do a lot more than just faffing around with how to arrange letters and word tokens).
> What I think the touchiness is about is that humans don't like the idea that a large part of their brain is an automaton
Whacking great chunks of our brains - well, include our entire central nervous system from scalp to toes - has all sorts of levels of automatic behaviour, from near-as-damnit literal automata (bonk that bit with a little rubber hammer and watch the muscles twitch) to unconscious feedback (e.g. touch preventing overcorrection of hand motions) to semi-conscious (e.g. sight causing overcorrection of hand movements - you can consciously observe it, just try consciously taking control of it) to - well, I'd love to say "conscious" but there is so much evidence that our brains do things, like deciding to move hands to type, before our conscious part realises it and just says "of course, I meant to do that all along".
I would like to think that El Reg commentards are well-enough aware of at least *some* of these aspects of our mushy internal goings-on that they are not frothing at the mouth any time someone tries to say that bits of us work on automatic.
PS
LLMs are not intelligent.
PPS
LLMs are not the be-all and end-all of AI; there are other areas AI covers than just this one application of one of its techniques.