Working as intended
This is it working exactly as intended, the original purpose of these LLMs was only to generate text that is stylistically indistinguishable from its training data. It does not and can not care whether what it's producing is factually correct. It's purely a side effect that they ever produce factually correct answers.