Back to ASICs: Mellanox pumps up Ethernet speed to 400Gbps

Just as we're getting used to 40Gbps, Ethernet networking kit-flinger Mellanox makes it 10 times faster with a Spectrum-2 ASIC running at up to 400Gbps. It's doing this because of our old friend incessant data growth. Specific drivers mentioned by Mellanox include AI, real-time analytics, NVMe over fabrics storage array access …

[Anonymous Coward]

By next year....

it will no longer be supported by them as they will have brought out something newer and faster like with everything mellanox do.

[Lior Paster]

Re: By next year....

That is completely untrue... Mellanox still supports NIC's sold 8 years ago, and Switches sold 8 years ago. Mellanox innovates and stands behind its products. If you have a problem, be specific.

[Anonymous Coward]

Re: By next year....

'support' and actually support by developing drivers that actually work for operating systems with features in the spec are 2 different things.

Take the connectx2 infiniband cards. No official support in esxi5.5 for over a year and after no support anymore for rdma protocols like srp/iser, same in server 2012r2, only 2008. No planned updates to do so. Stability problems that don't get fixed.

Only Linux gets these features and have actual proper driver support due to the ofed community.

[nilfs2]

What about the NICs?

Do they sell the 400Gb NICs and SFPs as well?

[chris 143]

Re: What about the NICs?

I don't think you'll be putting a 400GB nic in the back of a sever any time soon, largely due to the speed of the pci-e 3 bus.

[TheVogon]

Re: What about the NICs?

"I don't think you'll be putting a 400GB nic in the back of a sever any time soon"

A 400Gb nic however is quite possible using PCI-E v5 - that will be available within 2 years or so.

[Lior Paster]

Re: What about the NICs?

400GbE will be used for now to connect between switches, but around 2020 or 2021 PCIE Gen5 will actually enable 400GbE NIC throughput.

[H in The Hague]

The need for speed

"... up to 400 Gbps"

110 Baud was good enough for us.

Young people these days!

[Anonymous Coward]

We had to make do with 75 baud - but we were fortunate compared to those who had even lower bit rates. It was easy to jam the Teletype keys by typing too fast - even with two fingers.

IIRC the lowest I heard about was 1 baud. That was a low power radio rescue channel transmitter. The receiver had a very, very narrow filter. To keep the transmitter frequency stable it was designed so the downed pilot's armpit acted as a crystal oven.

[Korev]

Re: The need for speed

110 Baud was good enough for us.

Young people these days!

Yep, these "millennials" get baud so easily these days...

[thosrtanner]

Is there a 400G standard already?

[Hurn]

SFPs + Fiber = cost more than switch?

Proposed specs - official IEEE release expected in December

400 Gbit/s Ethernet

at least 100 m over multi-mode fiber (400GBASE-SR16) using sixteen parallel strands of fiber each at 25 Gbit/s[19][20]

at least 500 m over single-mode fiber (400GBASE-DR4) using four parallel strands of fiber each at 100 Gbit/s[21][22]

at least 2 km over single-mode fiber (400GBASE-FR8) using eight parallel wavelengths (CWDM) each at 50 Gbit/s[21][23][24]

at least 10 km over single-mode fiber (400GBASE-LR8) using eight parallel wavelengths (CWDM) each at 50 Gbit/s[21][24][25]

200 Gbit/s Ethernet

at least 500m over single-mode fiber (200GBASE-DR4) using four parallel strands of fiber each at 50 Gbit/s[26][27]

at least 2 km over single-mode fiber (200GBASE-FR4) using four parallel wavelengths (CWDM) each at 50 Gbit/s[1][27]

at least 10 km over single-mode fiber (200GBASE-LR4) using four parallel wavelengths (CWDM) each at 50 Gbit/s[1][27]

[Kernel]

Re: SFPs + Fiber = cost more than switch?

So, the obvious solution if you want to go much further than on-campus will be to use DWDM technology, where 100Gb/s on a single fibre, single wavelength is possible over a 1000km or more, and 200 and 400 can be done the same way up to a couple of 100 km.

I wouldn't be surprised if we didn't see terabit speeds in the long-haul space within a couple of years, although I have no inside knowledge of such developments.

[Meph]

Re: SFPs + Fiber = cost more than switch?

I'd personally be chasing the balance point between speed and reliability. Being able to push the signal further between repeaters will mean a reduction in at least one possible point of failure on long haul infrastructure. Lowering your mean time between failure (assuming no catastrophic natural disasters or idiots with a JCB and a loose and free attitude to trench digging) is a hell of a cost saving, and may offset the price of more expensive kit, at least to some extent.

Still... 1Tb+ long haul links and multi-gigabit speeds to the home does sound awfully good..

[nilfs2]

Re: SFPs + Fiber = cost more than switch?

"...at least 500 m over single-mode fiber (400GBASE-DR4) using four parallel strands of fiber each at 100 Gbit/s[21][22]"

So, that means that the highest possible speed for a single connection is 100Gb?

[Glen Turner 666]

Single-flow speed of nBase-xx4 links (was: SFPs + Fiber = cost more than switch?)

"So, that means that the highest possible speed for a single connection is 100Gb?"

No, you get 400Gbps. The nBase-xx4 interfaces run four "lanes" of ethernet symbols. The symbols are round-robined between the four lanes. An ethernet symbol is 64 bits logical, 66 bits on the wire (to allow for clock recovery).

If you are thinking that this means the media carrying the four lanes needs to have exactly the same latency then you would be correct. This is conveniently enforced using fibre assemblies and connectors with multiple fibres.

[TheVogon]

Re: SFPs + Fiber = cost more than switch?

"So, that means that the highest possible speed for a single connection is 100Gb?"

No - you missed: "at least 10 km over single-mode fibre (400GBASE-LR8) using eight parallel wavelengths (CWDM) each at 50 Gbit/s"

That's a single fibre connection.

[E 2]

Let me be the first...

to hail and welcome our extremely high bandwidth overlords!