Re: Not sure why misalignment happens
> aren't sophisticated enough to do the very best of human thinking.
More a case of the sophistication all being poured into one side of the equation and simply not bothering at all with the other side, the "boring stuff": as in, consider how many people want "an answer" but will glaze over - at best - when presented with the explanation of *why* that is the (or an) answer; the money (if there really is any, long term) is in pandering to the good old Lowest Common Denominator.
> We train ourselves to follow procedures and leave an audit trail of our own decisions
And AI researchers[1] aim to do just that, with mechanisms that, strangely enough, arose *after* the ideas of Neural Nets were dreamt up - so maybe we are just at the wrong period in time and the big money will start to be poured into ideas from the 1970s, 1980s and beyond rather than must building larger and larger 1940s and 1950s boxes!
The LLMs are "designed" not to leave audit trails - certainly not ones that are even vaguely useful. Even if they logged all the values flowing through the networks and printed them out - which they could (logically) easily do - turning that morass into anything comprehensible is beyond us; a raw audit trail of *any* large system has to be processed before it can be used by a human and we do not have any serious idea how to do that processing at that scale. Note that smaller, more constrained, 'Nets *can*, to an extent, be examined; for example (some of) those that process images can have their internal states turned back into images which we can then interpret.
In stark contrast to LLMs, Expert Systems have "self explanatory" as a core part of the design, the entire output of a Planner is - a plan that you can read, even before it is acted upon.
But making those work takes more than just buying more and more identical units, shovelling up more and more "input" without ever examining it - oh, and they have this pesky habit of being able to turn around and say "Nope, you ain't getting an answer to that, it outside of my scope" - not useful when you are trying to flog the Universal Solution.
"AI" as the field of study, not just the tediously single-topic-of-the-day reference to LLMs, aims to provide what the systems you desire. Please don't give up on the whole thing just because we aren't there yet.
[1] the proper ones, slaving away at the mercy of the funding boards, not the ones just shoring up the LLMs for The Usual Suspects
