Re: RAM
TB4 has a bandwidth of 80 Gbps, or 10 gigabytes per second. The LPDDR5 in the Studio with the M1 Ultra has 800 gigabytes per second memory bandwidth, or 80x faster. That doesn't even get to the orders of magnitude worse latency you get over TB4.
The reason accessing GPU memory over a PCIe bus (which is faster than TB4 though still far slower than the M1 Mac's memory bandwidth, and still quite problematic for latency) isn't a problem for GPGPU because when the CPU needs to access GPU memory it isn't latency sensitive - i.e. you are shoving a bunch of data over to the GPU for the GPGPU program to crunch on, or shoveling it back from the GPU for the CPU to operate on (i.e. save it, show it to the user, whatever)
If you ran a program on your CPU operating on memory in a PCIe attached GPU you'd think you'd been transported back in time 30 years from how slow it would run.
Some GPUs have a lot of memory bandwidth - Nvidia's latest announced yesterday even tops the M1 Ultra's. But that's only for GPGPU programs. Running anything on a CPU trying to access that memory over the PCIe bus it is like trying to fill an Olympic sized swimming pool with a 500 foot long garden hose.
Biting the hand that feeds IT
