"OpenAI spokesperson agreed that the differences between the models was down to the nature of the underlying datasets used to train them."
... and that's exactly what this is. In order to train a complex model, especially generative one, you need lots and lots of data. If the input data set is not carefully screened then obviously it will contain biases, which in turn will influence the models. Screening of the input dataset might not be an option because of the sheer volume of data required. You would either end up having to rewrite input texts to remove biases, which would take ages; or have much smaller dataset for training which would make it impossible to train such complex models.