back to article Supermicro breathes in, shimmies a PB of Intel flash into one rack unit

Supermicro has crammed 1PB of Intel flash rulers into the slimmest possible 1U rack storage server. The two-socket server can hold up to 32 Intel EDSFF, NVMe-connected flash drives – giving a rack density of 1PB/U, the highest we have ever come across. EDSFF stands for Chipzilla's Enterprise and Datacenter Storage Form Factor …

  1. ColonelClaw

    What's the deal with NMMe flash drives and hardware RAID controllers? I can't find any!

    1. Ian Michael Gumby

      @Colonel Claw

      Yeah, you can't.

      I think you may want to try CDW but even there... not much.

      I think this is something you have to go directly to your hardware vendor to get, and then maybe some of the vendor supply shops that usually don't deal directly with consumer.

      Very pricey and most of these guys won't keep kit on hand and will have to special order it.

      My guess... the 1U box would be over 500K (USD) to 1 million (USD) fully kitted out. (Probably closer to 600-700K range.

      One of these in the center of the rack w 4 4U boxes below and 4 4U boxes above. (plus ToR switches)

      And the 4 U boxes contain NVidia GPUs.

      Now that's a killer platform for ML or 'Big Data' with the 4Us also having a bit of local disk too.

      Of course you'd be looking at a cool 2 million+ USD per rack fully kitted out...

      1. Anonymous Coward
        Anonymous Coward

        Re: @Colonel Claw

        Imagine the rebuild time to your application if you lose a node... 1PB..... Says 25 days? No biggie.

        1. Ian Michael Gumby

          @AC ...Re: @Colonel Claw

          Naw, not really.

          There's more to the design than just a single rack.

          Also we never talked about DR/BCP planning and your second site.

          There's also a bit more to the design... but I'll leave it to you to try and catch up first.

        2. fredesmite

          Re: @Colonel Claw

          Stack three together using Ceph and you have self healing storage needs .

    2. baspax

      No RAID for NVMe

      NVMe has no SAS stack and hence no RAID. You have a bunch of individual devices which are very very fast. Usual limitation is 10 per servers due to lack of PCIe lanes.

      I need to take a closer look at this box and see how they upped the number of available lanes for a 2 socket system, if they didn’t it will be a world of pain.

      This box needs a fast SDS layer to control all those individual devices, VSAN and Nutanix for example.

      1. Michael Duke

        Re: No RAID for NVMe

        Well AMD Epyc has 128 PCIe lanes in either single or dual CPU configs.

        32 Lanes for external comms (4 x 100GbE or 16 x 25GbE ports) and 96 lanes for 32 storage devices (3/Ruler) makes sense. That is nearly 3GB/Sec per device (2,955MB/Sec).

        Of course Intel will probably not license the EDSFF form factor for AMD based systems so it might all be for naught.

    3. Roj Blake

      Re: hardware RAID controllers?

      Intel do a thing called VROC - Virtual RAID On CPU. It's designed for use with MVMe and while it's not quite as good as a hardware RAID, it's a lot better than software.

      1. fredesmite

        Re: hardware RAID controllers?

        does it cure the write hole ( multi block rewrite ) problem ?

    4. Michael Duke

      Re: This is Dell or EMC?

      You will not find a traditional RAID card that can keep up.

      Core CPU driven Scale Out Software Defined Storage will be the norm for these.

  2. eswan

    That's alot of porn.

    1. Ian Michael Gumby


      Hmmm... 8K resolution?

      I wonder how much you could store uncompressed.

      1. 9Rune5

        Re: eswan

        I wonder how much you could store uncompressed.

        Depends. Gay or straight pr0n?

        Besides, this 1PB storage unit classifies as hw pr0n. So... If you store pr0n on a hw pr0n device, will there be a rift in the space-time continuum? A 'pr0n hole' if you will.

        1. Anonymous Coward
          Anonymous Coward

          Re: eswan

          A 'pr0n glory hole' if you will.


  3. Borg.King

    As exciting as this is

    Is it economically sensible to actually buy these compared to, say, four 256TB units and then spend the purchasing dollars saved on renting extra datacenter floorspace/power/cooling?

    I'm sure in time they'll become economically viable, but is that 6 months from now, or 2 years?

    I'm not a datacenter purchasing, commissioning or operations engineer (I write code), just curious to understand the economics here.

    1. irrision

      Re: As exciting as this is

      It's about density and you've got a workload that needs many petabytes then you'd go with something like this versus a number of smaller units. Also the incremental cost for each server, plus any licensing might be another 10k a unit though that's probably a drop in the bucket compared to the total cost of one of these loaded up with flash.

  4. This post has been deleted by its author

  5. Twanky Silver badge
    Thumb Up

    I'd buy that for a dollar!

    "El Reg has yet to receive pricing and availability information for Supermicro's latest 1PB 1U box, but anticipate the $/U rating will be on a par with the PB/U value."

  6. p3ngwin1

    Title Gore...

    You mean "Supermicro breathes OUT...".

    When you breath IN you take IN air, expanding yourself larger.

    When you breath OUT, you get smaller :)

    1. TRT Silver badge

      Re: Title Gore...

      But the phrase goes "Breathe in!" And I think that's related to fitting a corset or belt where the intercostal muscles contract and lift the rib cage up, allowing bits of viscera etc to squeeze upwards leading to a smaller waist.

  7. J. Cook Silver badge

    ... I need four five of these. And a betting pool to see how fast certain departments at [RedactedCo] will take to fill the damn things. :D

  8. fredesmite

    Are these ssd simply block storage ? Any SDS component to them ?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021