Re: time diff
Yeah, it's not clear from the summary in the article what exactly the test was. I expect it was actually in effect "find the passage in the context window that doesn't match what you would expect for the next token", so it wasn't diffing Real TGG against Modified TGG; it was running Modified TGG against the entire model, which included somewhere in its parameter space a gradient matching Real TGG but not the actual text in a literal representation.
Not a hugely interesting experiment, as far as I'm concerned. Exactly what I'd expect a really large model to be able to do. So what?
Now, if the underlying model had been trained on a data set from which all copies, excerpts, and references to Real TGG had been removed, and it still caught the offending passage, that would be a slightly more interesting experiment. (It's feasible for a transformer LLM to do this, if there's enough similarity between the world of the novel and the world of the training set for most-probable completion to get a strong disagreement on the altered passage.) A better test would be to use a freshly-written unpublished novel, of course, so there's no possibility of data-set contamination. But even then, all you've confirmed is that the surface of parameter space contains a gradient that diverges sufficiently at the point where the out-of-place passage appears.
And that's a big problem with LLMs. They converge on a middle ground of expectation. They seek to reduce surprise, which is another way of saying they reduce information entropy in the output. They're bland. They have no style. They have no conversation, as we used to say of uninteresting people. They regurgitate the most likely continuation, in a dull fashion. You can anneal them into slightly higher valleys with prompting, but the existing models and their architectures fundamentally lack the inconsistency of human discourse. And that's what makes us interesting.
Biting the hand that feeds IT
