AMD targets Nvidia H200 with 256GB MI325X AI chips, zippier MI355X due in H2 2025 AMD boosted the VRAM on its Instinct accelerators to 256 GB of HBM3e with the launch of its next-gen MI325X AI accelerators during its Advancing AI event in San Francisco on Thursday. The part builds upon AMD's previously announced MI300 accelerators introduced late last year, but swaps out its 192 GB of HBM3 modules for 256 …
MI325X looks pretty good imho. 256 GB of HBM is huge, and a proper base-2 figure, which seems to fit expectations better than 288 GB. Then again, 288 GB might have been a datacenter-oriented "target" that includes ECC bits, say 72-bits per stored item (8-byte data + 8-bit ECC) x 32 G locations (the basic storage cell being effectively 9-bit wide rather than the more common non-datacenter 8-bit). That sort of memory is more commonly paired with CPUs though iirc ... (and might be referred to as 256GB with ECC, rather than 288 GB).
I wonder why they don't just make a product with more previous gen memory? Surely there is a market for a slower card, but with more VRAM? Not everyone is training AI models...but a hell of a lot of people want to run them...something based on an older chip with a lot more VRAM targeted specifically at running models seems like a no brainer to me.
I can't see this sort of product cannibalising the sales at the top end, because the organisations at the top fighting the arms race to produce ever more capable models are running a race that most mortals cannot fathom that depends on having cutting edge performance...
...but those consuming the models don't require cutting edge performance, they just require more VRAM...something roughly the power of a 4070 or 4060 with 64GB of VRAM would fly off the shelves for dedicated LLM builds.
AMD releases something new, and supposedly "threatens Nvidia".
Nvidia's advantage is in their software as much as their hardware. AMD could beat Nvidia on performance and Nvidia will still dominate the AI market because everyone is using their software. AMD would have to perform 50-100% better before the market would take notice.
Not really. There are already translation layers to enable CUDA code to work on AMD GPUs...
The advantage NVIDIA has over AMD is they saturate Universities with their kit and insist on using their software with them if they want support...which unleashes loads of brainwashed people into the real world that only know CUDA.
Once we move away from AI being largely academic to being largely commercial, there will be a large shift. It's already happening to a certain degree.
There are AI libraries out there that AMD absolutely mops the floor on when directly compared with NVIDIA...the problem with CUDA is that a direct comparison cannot be made. I you do compare NVIDIA on CUDA to AMD on something else, the performance gap isn't that massive. It is significant, but not significant enough to justify the cost.
Pretty soon, performance won't really matter all that much because AI will reach a point where precision becomes more important than performance, because beyond a certain point, performance stops being important, but accuracy becomes more important.
At the moment, if you want accuracy, you need to run larger models, which is easier on AMD hardware due to more VRAM than NVIDIA hardware...NVIDIA kit might produce faster results, but AMD kit right now can produce more accurate results.
I'm not a fan boy of either side because I think both sides could do a lot better than they are and both sides have their inherent problems...but the AI race will eventually be won by AMD I think...their kit has always had more raw compute available and when solutions start moving to non-proprietary backends we'll see that extra compute making contact with the tarmac and at that point, there will be a whole back catalogue of used AMD kit that will be dirt cheap.
Right now, I currently have an AMD card in my dedicated LLM box...purely because it allows me to run larger models, I have the NVIDIA card in my desktop.
Is it slower...yes...is it slower to the point that it makes a difference to my workflow? No, not at all...with either card, it produces results faster than I can read them...therefore it came down to whether I could run larger models...AMD cards can do that, NVIDIA cards, currently, cannot.
The next generation of NVIDIA cards will likely have loads more VRAM to try and combat this, but historically, AMD has always leapfrogged NVIDIA with VRAM capacity so I don't expect NVIDIA to increase their VRAM above that of AMD...if the flagship NVIDIA card comes with 32GB of VRAM, the AMD flagship will have 48GB...purely because NVIDIA can't help itself when it comes to cutting corners and min/maxing profit.
Right now, if the technical complexity isn't a factor for you, the choice you have is:
1) Run slight older, smaller models as fast as possible...NVIDIA.
2) Run the latest, larger models...AMD.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
