Re: Being able to make the Distinction
"LLMs on the other hand, are identical buckets of bits running on deterministic silicon. Every instance of an LLM is the same pile of statistics, i.e. the same pattern of connections and weights, with randomness added only as noise in one or more of its layers. Even if that randomness could be quantum randomness, it cannot adapt its own weights on the fly, and both its weights and state are known. If you could show me an LLM where each INSTANCE had its OWN weights, that were all quantum-unknowable and self-adapting on-the-fly, then I might be able to call it something like alive."
I'm happy to be corrected if I'm wrong [in fact, please *do* correct me if I'm wrong]... but I think the above may be subtly but significantly incorrect...
All of the current generation of AI models essentially go through two stages of pre-use development. First, the basic model is "populated" - the code is developed, the underlying data structures are created [the digital neurons, for example]. Then, the model goes through training, of which there are at least two main types [open and closed]. Once the model has been trained, the model weights are locked down and the model itself is released for use. You *have* to lock down model weights before releasing for general use, otherwise it would be possible for your users to corrupt and pollute your models.
This means, in other words, that your opening statement, "Every instance of an LLM is the same pile of statistics, i.e. the same pattern of connections and weights, with randomness added only as noise in one or more of its layers", cannot be universally true. If I purchase an empty instance of ChatGPT and train it by feeding it legal briefs, so as to build an aid for the legal profession, I am going to end up with a different model than if you took the same version of ChatGPT and trained it by feeding it papers on quantum physics.
But, by definition, discrete instances are almost certainly going to have different weights and therefore will produce different results.
Somewhat orthogonal to this, but something that can produce converging results, is the use of context windows... When you start a conversation with a model, it will have a set amount of "context" [capacity to store and refer to a tokenized history of your conversation] that it can rely on during your interaction. However, because that context size is finite, when your session with the model gets close to filling the context window, it will invisibly and automatically perform a "digest" - make a summary - of your conversation. If you are performing a repetitive task with a model and you give it an initial set of instructions and then sit feeding data in to it, then the moment that your session experiences it's first context compression/digest, you will inevitably start to get subtly different results than you would have received if you were interacting with the same model in a "pre-compressed" state. Worse, the further you go - i.e. the more times that you force your context to perform another digest - the more risk of "summarization drift" you risk adding to your model... and the greater the deviations you might see from the results.
Frustratingly, there doesn't seem to be any easy way to "know" if a session you've got running has been through a digest cycle...
But in my very limited experience, I've found these intrasession maintenance activities to be more impactful on the quality and consistency of the results than pretty much anything else.
Footnote:-
There's one other thing I've seen [ from Anthropic Claude Opus] in some conversations... which is that you can ask it for input and it will go away and come back with something which is fundamentally wrong... but when you challenge it, the model will glowingly tell you that you're a genius and then give you the right answer... If you sidestep the flattery, I *suspect* that what is happening under the hood is that the model has been "profiled" to only spend a certain amount of effort answering a question - *even if that means giving a wrong answer*.
Two glaring examples I've seen: in a chat concerning some reasonably deep theoretical physics, Claude tried to tell me that gravity was a force. When I countered that General Relatively shows that gravity is in fact a property of spacetime, Claude immediately agreed with me and then went further with the answer I'd been asking for... ; second, in a session where I was using Claude Code to make some updates to some PHP I've written, it correctly identified that the code needed an authorization check built in to it... but then proposed to add this to called functional logic rather than handle it once in the dispatcher [where access to called logic could be outright blocked]. In the second case, Claude's response was to the effect, "You're right - that's a much neater, stronger solution, for the following reasons..."
Which rather leads me to wonder that there is a lot of "pulling the wool over the eyes" being coded in to the models by the various companies offering them.
It also leads me to speculate that they are not disclosing this and not discussing their motives... but that this is most likely being driven by attempts to reduce the operational cost of running the models - by "forcing" them to limit the effort they spend on answers. This in turn leads them to go with an answer that's easy to find rather than the best answer. That in turn means that if you want to get the best out of a model, you actually need a pretty good level of knowledge of the subject yourself.
All of which is a long-winded way to say that first: I'm not convinced that weights are uniform [or even should be]; and second, that there are very clearly other non-published parameters capable of having just as much influence over the output and which we are presently not being made aware of...