Re: Is this not just how they work?
Your argument requires that the products people can buy have a "stated design specification". What is the "stated design specification" of ChatGPT? All the text the average user sees defining what it is is from marketing, not a technical spec. The reason is that there isn't and never has been a specification for what that product is, not from the creators, not from the marketers, because they don't have one. One of their components is a thing that guesses words, but that's not the only part in the product people are buying. They're buying the whole thing.
To continue the toaster metaphor, one of the parts in that does not have the stated design specification of "make toast". The part that does most of the work has a stated design specification of "produce heat when electricity is passed through". Whatever you do with the heat from that part is your decision. The customer expects the toaster to make toast because that's what the whole system does. Complaining that it doesn't is not fixed by pointing out that the heating element does exactly what it was intended to do. Those developers who build this software fall into two groups:
1. Developers who primarily work with the word guessing part and are talking only about it. These people are not lying, but their papers about the word guessing part are not talking about what people are actually buying.
2. Everyone else, who knows full well that what they're telling people their software does is not what it actually does. There are many people employed to work on other components of an LLM-based system. They don't get to argue that customers are just buying a word guesser when they're specifically employed to make it more capable. And, some of the time, they do. The word guesser is rather bad at doing any mathematical calculations, but some wrapper around it can make it capable again. Not efficient, but at least correct. That's a thing they intended and a thing they sell. That is part of the stated design specification to the extent they have one. Developers know this, and that includes developers of the word guesser part. As soon as they make a point relevant to the product rather than just that component, they are as culpable as the rest.