How Nvidia learned to embrace the light in its quest for scale

If you thought Nvidia's GB200 rack systems were big, CEO Jensen Huang is just getting started. At GTC last month, the world's most valuable company revealed plans to use photonic interconnects to pack more than a thousand GPUs into a single mammoth system by 2028. The company isn't waiting to secure supply chains either. Over …

[that one in the corner]

Copper is cheap

That is one of those relative measurements, isn't it.

(Looks at copper-covered aluminium jumper wires that won't solder to PCBs and copper-coloured steel ones that keep getting stuck to magnets)

[Eric 9001]

Re: Copper is cheap

Thin wires of copper is cheap, but CCA and steel is cheaper.

Aluminum will "solder" in a through hole or on a pad if you try hard enough.

[Anonymous Coward]

Sounds good

The linked "Next Platform discussed" piece has a nice slide of Celestial's Photonic Fabric Scale-Up Networks that claims a reach of 50 meters, bandwidth of 100's of Tbps, and 2.5 pJ/bit for the E-O-E link. That compares well to 7 pJ/bit for HBM and 25 pJ/bit for DDRx (from BSC and ETH Zürich, 2021) and should certainly help with building some "coherent memory network spanning multiple racks", as well as make very large numbers of XPUs "behave like one enormous AI [or other] accelerator" imho.

Plus, with 50m of reach, one could imagine central switch cabinets surrounded by a great many XPU racks in a circular architecture, possibly inspired by nature, as in noteworthy medical clinics (for efficiency; 2400 Ferry Street, Lafayette, IN), or by other strinkingly inspirational buildings, if desired (fewer positional constraints relative to copper's shrinking range at high signalling rates)! ;)

[anthonyhegedus]

We have been promised the light since the 1980s

Photonic interconnects have been the next big thing for about as long as I have been in the industry. Nvidia betting on them tells you less about optical maturity and more about how utterly we have exhausted what copper can do at these bandwidths. A thousand GPUs worth of data moved by light sounds brilliant until you remember every photonic link still needs electrical conversion at each end. The bottleneck does not disappear. It just gets a more expensive postcode.

[Anonymous Coward]

Every hardware announcement by NVIDIA reduces their GPU output for gamers, sending prices ever higher, so I take these press releases with huge grains of salt. Salt that they rub into my wounds gleefully, so their investors can make a few million more.

RAM costs are skyrocketing because of short supply and GPU are becoming as rare as Vaquita. If photonic connectivity helps get more GPU out the door, cheaper, I'm all for it. Otherwise, we're just seeing more toys for the mega-rich while the rest of us are left out in the cold.

Gamers used to be their life-blood, but they just don't need us anymore. I'd go AMD, but they are starting to twist the knife as well.

[khjohansen]

"Gamers" are a diminishing market share

When I tell people I used to build my own desktop PCs, they give me looks - the gamers of today are on mobile or consoles!

(I know, these platforms need GPUs as well, but don't carry the premium or the replacement schedule of video cards)

AI/LLM, on the other hand, has massive venture capital ...

[Anonymous Coward]

This is going to be a nightmare

I've had the pleasure of working hands-on with CX8 optics in... pretty much this setting, and it's nothing short of an unmitigated disaster. Failure rates are quite high on the transceivers. If you build those into the board and GPU fabric it's going to be difficult to work with to say the least. There's a good reason the optics are packaged separately. I was told by Nvidia engineers themselves that they couldn't get optics to handle nvl72, and I really doubt they're going to actually get a useful implementation in the future.