@LionelB
>> It's not even useful as a computer, because it doesn't spit out facts
> Wait... you thought that's what computers do?
Computers spit out deterministic answers after performing calculations on their input data. They will always produce the same answer for the same input data and the same "question"/formula. This could be a database query, for example. If I ask MySQL to give me the rows of the database matching my query, then it is a "fact" that those rows exist in the database. If I ask the same of ChatGPT, then it is a mere statistical likelihood that rows such as these might have existed in some database, once upon a time in fairyland.
The key here is determinism. ChatGPT's answers are based on randomness, and its output modifies its input. It does not give the same answer for the same question. It cannot reliably produce a factual output, even if its answers are sometimes, even usually, correct. Whereas computers running deterministic programs can be relied upon to produce facts, as long as their input data is correct of course.
> Like a student essay, then.
No, because a student can answer questions about his/her essay and will have learnt something from the act of writing it and answering the questions, which was the only point of the essay. If you gave ChatGPT an essay that it wrote, it will have no awareness that it ever produced that output, and can merely pretend to answer your question.
> Chat GPT (in my experience) stubbornly refuses to introspect or discuss itself.
Because (news flash) ChatGPT is NOT self-aware. Not even when it eventually does incorporate all of its past outputs into its input will it ever be self-aware. It is not alive. There is a hell of a lot more that a real organism does to make it alive than the language model that ChatGPT uses.