back to article AMD scores 200,000 cores worth of secret silicon at new Australian supercomputer

Australia’s Pawsey Supercomputing Centre has bet on unannounced AMD Epyc silicon to power a new 50-petaFLOP supercomputer. The centre refused to reveal clock speeds or other details of the silicon to us but did say that it will comprise over 1,600 dual-socket nodes and utilise over 200,000 cores. Those figures yield a per- …

  1. Korev Silver badge
    Coat

    HPE Slingshot interconnect tech

    Hang on, don't the Aussies use Boomerangs?

    1. Chris the bean counter

      2 Boomerang facts

      1) The biggest boomerang factory in the world is in WImbledon

      2) The name for a boomerang that does not come back is not a stick...its a kylie (honest)

      1. Tom 7 Silver badge

        Jack Cohen was a mate of my dads and IIRC was one of the first people to bring the boomerang to the UK and try and make it a hobby. Remember many happy hours running around fields chasing badly thrown ones. Never ceases to amaze me how a piece of wood you threw away 30 seconds ago still has enough kinetic energy to make your fingers burn for hours!

        1. Dwarf Silver badge

          @Tom7

          Now I need to go and watch Max Max 2 again

  2. Alan Brown Silver badge

    1600 systems at around 600W at full song (sans GPUs) = 960kW

    That's "only" 120kW per cabinet, which puts my 12kW power budgets into perspective (and i was being told off for thinking we'd need more than 5kW per rack...)

    now where are my marshmallows?

  3. Phil Kingston

    $6M per rack. I'm sure someone can do a comparison for me against, say, a similar volume of maybe gold? Just curious.

    1. eldakka Silver badge
      Boffin

      $6M per rack. I'm sure someone can do a comparison for me against, say, a similar volume of maybe gold? Just curious.

      Challenge accepted!

      At time of writing the spot price of gold (troy ounces):

      AUD$2,708.30

      So $6million/2,708.30 = 2,215 ounces (troy) of gold

      According to wikipedia:

      Gold bars

      Fineness: minimum of 995.0 parts per thousand fine gold

      Marks: serial number, refiner's hallmark, fineness, year of manufacture

      Gold content: 350–430 troy ounces (11–13 kg)

      Recommended dimensions

      Length (top): 210–290 millimetres (8.3–11.4 inches)

      Width (top): 55–85 millimetres (2.2–3.3 inches)

      Height: 25–45 millimetres (0.98–1.77 inches)

      Therefore you'd need 5.15 13kg gold bars for $6million. Making the unfounded assumption that the largest dimensions are those of the largest (13kg/430 ounce) "Gold Delivery" bars, that'd be a row of bars 290mm long, 437mm wide, and 45mm high (5.7 litres volume).

      A 19-inch rack is 44.45 mm per RU and 482.6 mm wide (19 inches duh), and arbitrary depth, 600mm, 800mm, 1000mm, it varies. So, apart from being 0.5mm taller, that much gold in Gold Delivery gold bars takes up less than 1/2 a 600mm deep 1RU rack mount slot ;)

      1. Phil Kingston
        Pint

        That is the kind of answer I come here for. Have a beer.

  4. Ubermik

    Poor old Intel.....

    AMD seems to be snapping up a huge share of the huge compute market over the last few years

    Does intel even have silicon that comes close to the ROME CPUs yet? As the next gen is apparently more power efficient with around a 20% IPC gain and much lower latency

    Intel could take half a decade to catch up and thats assuming they even can catch up if AMD keeps having such large leaps with each generation

    If I recall, 1 64 core Rome CPU wipes the floor with a dual socket top silicon intel offering whilst also being about half the price and taking MUCH less power

    In another couple of years the remaining intel customers are likely to look like people who were still clinging to their Atari STs in 2010 claiming they were still superior to anything the PC had to offer for gaming lol

    "For those OCD sufferers who would freak out if their CPU didnt match the "intel inside" sticker on the case there are intel CPUs,,,, for everyone else there is AMD :)"

    1. eldakka Silver badge

      Re: Poor old Intel.....

      If I recall, 1 64 core Rome CPU wipes the floor with a dual socket top silicon intel offering whilst also being about half the price and taking MUCH less power

      If you need large memory support, it's worse than double for Intel equivalent. The prices I've been able to find (Inte/AMD don't tend to make list prices available to the unwashed masses for their high-end server processors, so can be hard finding accurate prices) are for the AMD 7742, 64 cores, 2.25GHz-3.4(?)GHz, ~US$7k. I couldn't find a price on the 2.6GHz-3.3GHz 7H12.

      The nearest equivalent Intel processor is the Platinum 8280, 28-cores, 2.7-4GHz, ~US$10k. But note that it only supports 1TB of ram per socket, whereas the AMD Epyc support 4TB per socket.

      So a 2-socket (the maximum Epyc supports) 7742 would be 128 cores, up to 8TB RAM, $14k for the CPUs.

      For Intel you'd need 4 8280's to get to 112 cores, them being a bit faster should make up for the core count deficit to AMD, for ~$40k. But only up to 4TB total system support (1TB per socket x 4).

      To get 8TB on Intel and the 112 core count, you'd need 4 8280Ls, which are $17k a pop. So 4 of these would give the same 112 cores (at the same speed of the 8280, the only difference between the 'L' and non-L is the memory size support), and the 8TB support (actually up to 20TB using Optane Memory), for $68k in CPUs alone. Five and a half times the 128 cores of AMD Epyc CPUs. And of course, being a 4-socket board, the board itself would be significantly more expensive than the AMD 2-socket board.

      Although I understand that motherboards that support 4TB per socket are very rare, hence Intel's deliberate segmentation of their processors into 1TB/socket and 4.5TB/socket lines. Most sales are for 1TB/socket, with customers wanting greater than that being fairly niche. But since AMD can far surpass Intel's core counts per socket, they'd obviously need to up their per socket memory support to allow greater overall RAM with less sockets.

      1. Ubermik

        Re: Poor old Intel.....

        If the hype is true then the next iteration cores also beat intel on a per core performance too, so (assuming its true) intel would probably actually need 140 ish cores just to pull even this time around

        Its like kicking a dog when its down because you just backed your car over it 5 times after dropping it onto concrete from an 8th floor balcony (bad doggy)

        I am expecting intel enterprise execs to be found on the client list for a seedy dungeon catering for the excesses of masochists :)

    2. Korev Silver badge
      Boffin

      Re: Poor old Intel.....

      A few years ago AMD were wiping the floor with their Opteron chips and then Intel stopped messing around and developed some better chips. IIRC Opteron's advantage finally ended completely with Nehlem's onboard memory controller.

      1. Ubermik

        Re: Poor old Intel.....

        Which really just showed that they could ALREADY have been offering customers better CPUs for the same or less money but CHOSE to rip them off instead purely because they could....

        You know, like they have bee doing with the 4 core cap on desktop CPUs for almost a decade, then the moment some competition comes along they "magically" discover how to have 6, 8 and then 10 cores on the exact same node

        Go figure

  5. Lars Silver badge
    Linux

    Linux on super computing

    While it's not the year of Linux on the desktop for everybody it has been the year of Linux on super computers for quite some time so that Linux is hardly ever mentioned.

    Compare that to the immense fuss when for a short time there was one running some Apple os.

    On this top500 statistics:

    https://www.top500.org/statistics/list/

    try Operation System Family for Category, it is 100% Linux.

    In November 2010 it was 92%.

  6. John Savard Silver badge

    Not too secret

    Basically the same silicon as will be in stores this November 5th, just with a few business features enabled, and clocked a lot slower. Just like last year's EPYC.

    So it's not too mysterious what the silicon they'll be using can do, or is like, but maybe AMD will come up with some little surprises. I'm worried that given that the new Ryzens need fancy cooling where the previous generation didn't, the clocks on EPYC - especially a version with more cores - will actually be slower this year compared to last. Not enough to lose all the improvement due to the IPC gains, but still... so if AMD manages to avoid that, it will be a pleasant surprise.

  7. VexedGuru

    And meanwhile in good old blighty

    .............the tumbleweeds rolls silently across our bleak supercomputing landscape

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021