Re: I don’t understand A.I.
This is one of the reasons for the DRAM shortage. (the main one being speculative construction of datacentres)
If everyone wanted to run their own 88GB model offline, then we would run out of DRAM even faster than we are now. It's more "efficient" to keep the same model loaded into RAM, running bullshit-generation for multiple "customers", on GPUs, with a load-balancing system to ensure optimal utilisation. (Yes, a CPU works adequately for a single customer i.e. you, but would be useless for a cloud AI server wanting to maximise responses-per-second-per-dollar).
The secondary "benefit" to people who operate AI in the cloud, is that they get to spy on everything that their customers are doing, which allows them to gather more training data, but also the majority of these users apparently don't care if the operator also sells the data to data brokers (who will sell it on to advertisers, scammers, spy agencies, other AI operators etc.)
> So, what need is there for all these data centres? Surely the Training is accumulative? If you need a more sophisticated model, you take an existing model and add more to it?
Because LLMs and indeed all Artificial Neural Networks are just averaging machines, simply adding more data often doesn't help, because any bad data already ingested is skewing the average and ruining the output. They don't really 'learn' in a logical reasoning way like humans (can) do. And if you want to -remove- any data post-training (either because it is bad, or because you got sued) it is not possible to completely remove it without training again from scratch.
And, if you want to change any design element or any of the so-called hyper-parameters, often you have to train from scratch anyway.
And often you have AI models doing nothing but generating training data for smaller, supposedly more-efficient versions of themselves, with added censorship or whatnot.
It's just bullshit all the way down, really. A massive waste of energy at a time when energy is in short supply.
