Energy Use
> You can run the models yourself, at home. On pretty limited standard PC hardware.
Er, that's true of most huge GenAI models, including Facebook's LLama. They take a "large" model that has been trained at great energy cost and would require massive memory/compute resources to run the "full" model, but they reduce the parameter count via "pruning" and quantize it down to FP4 from FP16, resulting in a shit version that will run on a raspberry pi. Nothing new there. I'm sure it could be done with GPT models too, but OpenAI choose not to, for commercial reasons rather than technical. (I have come to learn that the "Open" in "OpenAI" is meant sarcastically)
But when I say that the Chinese may have lied about their energy use, i'm not talking about the running cost so much as the training cost. If they have trained it using outputs from other LLMs, then they need the energy both for training a 671 Billion parameter model (a lot) and also the energy for generating 14.8 trillion tokens using the source models (a lot)
As I said in my earlier post, using LLMs to generate training data for LLMs is horrendously inefficient. But if you want to copy someone else's LLM and add your own censorship, all you need is a big stack of GPUs and a hell of a lot of energy.
Why would they lie? Well because a) it helps to wipe trillions off of western tech stocks and b) if they told the truth about the GPUs that they used to train it, it might prove that they are evading US sanctions