back to article Los Alamos National Lab fires up 750-node RPi cluster

The Los Alamos National Laboratory will this week reveal its latest "High-performance computer" - a cluster of 750 Raspberry Pis. The Lab's Gary Grider had the idea of trying to package up serious clusters, not to replace its supers – the three fastest at the moment are the Grizzly, Fire, and Ice systems, all in the 1-1.5 …

  1. Anonymous Coward
    Anonymous Coward

    Not enough buzzwords

    You need to add Cloud, Big Data, AI, Robotics and PaaS (Pi As A Service) before you can market this.

    Noboby in management, who holds the budget purse strings, will buy it based on how fast or useful it is.

    1. Anonymous Coward
      Anonymous Coward

      Re: Not enough buzzwords

      It is designed *not* to be fast or useful in itself. It is for developing cluster software on.

      1. John Robson Silver badge

        Re: Not enough buzzwords

        “It is designed *not* to be fast or useful in itself. It is for developing cluster software on.”

        It is designed to be useful - a cheap dev environment is *very* useful

        1. Alistair

          Re: Not enough buzzwords

          "a cheap dev environment is *very* useful"

          So -- we should be calling it Thomas then?

    2. A Non e-mouse Silver badge

      Down votes

      I think some peoples sarcasm detectors need re-calibrating this morning...

    3. M7S

      Re: Not enough buzzwords

      Ok then




      Testing on


      Research &


      1. Anonymous Coward
        Anonymous Coward

        Re: @M7S Re: Not enough buzzwords

        OT CUSTARD?

        Seems to be missing an H at the begining....

    4. MyffyW Silver badge


      Sorry, but once you ruled out Lego, all the enthusiasm drained from me

  2. A Non e-mouse Silver badge

    Big Pie

    Still needs a bit of work to catch up with Steve Fuber's Spinnaker ARM cluster. At the last count upto 500,000 ARM cores.

  3. Lars Silver badge

    Funny how

    Many years ago regarding large computers Linux was never mentioned and now again it's never mentioned as it has become the default. On the top500 list more than 50% ran Linux before they started to count them. I don't give a shit, sort of, but it certainly shows the importance of marketing. Once, for a very short time, there was one running "Apple" and wow the ink about that. Just imagine if MS payed somebody enough to try to tweek Windows into the list. Sometimes I have this feeling that if there ever is Linux on the desktop (19 years for me) it will happen in a similar way, no fanfare, most cars have four tires, ever told anybody about it.

    In a way Windows and Linux are very similar, if there is a new virus Windows is never mentioned, it's the default.

    1. phuzz Silver badge

      Re: Funny how

      People don't really use desktops much anymore, the computer they use the most is the phone in their pocket, and the majority of those run Android, which is based on Linux.

      Of course, it's using linux to run proprietary Java apps so it's not exactly what the open source zealots were expecting...

    2. Anonymous Coward
      Anonymous Coward

      Re: Funny how

      Microsoft had (has?) a variant of Windows Server for HPC. A few places played around with it but it seems to have gone nowhere since.

  4. Mikel

    Probably the Pi Compute Module

    It's a Pi on a SoDIMM. This is what it's for.

  5. druck Silver badge

    Pi flavour?

    [Edit: as Mikel asks;] What flavour of Pi's are they using? Full Raspberry Pi 3B's, Zero or more likely compute modules?

    1. Return To Sender

      Re: Pi flavour?

      Full sized Pi:

      1. jabuzz

        Re: Pi flavour?

        Anyone using an actually Raspberry Pi for a 750 node cluster is an idiot. It would make way way more sense to use a Orange or Banana Pi and get an actual 1Gbps Ethernet connection instead of dicking about with a 100Mbps one. It's a 10 fold improvement in one of the most critical components in a Beowolf style cluster.

        The sensible thing is to stick 48 Orange/Banana Pi's in a 1U and marry them up to a 48 port GB switch, and then tie 16 of those together with a 10Gb switch for 768 nodes. That's 32U of rack space leaving enough left over for a more normal x86 server to act as a file server.

        Oh and my day job is an HPC system administrator...

        1. Nate Amsden

          Re: Pi flavour?

          I certainly could be wrong (never used any Pi ) but I thought I had read the ethernet on the Pi was running off the USB bus ?? (not sure if still the case), but as you say, probably not a very good setup beyond a simple toy - the exception may be for setups that aren't network bound (e.g. download a batch of data to work on and then work on it from local storage/memory).

          Even if it's only 100Mbps, as long as it's on the PCI bus (not USB), I'd think would be a major improvement over anything running on top of USB.

          1. James Hughes 1

            Re: Pi flavour?

            Yes, the Ethernet and 4 port USB hub all go via a single USB port into the SoC. Yes, it is a bottleneck, no it hasn't really stopped a lot of sales, because in general people are not too worried about it. Those who really need the throughput get alternative devices and live with the less than useful support.

            As for whether this is a toy, I'd suggest that if Los Alomos National Lab though it was a good idea to make one, then it's not really a toy. After all, it's not like the Pi is the new kid on the block with an unexpected design that will catch them unawares.

            This is a project for testing code in an environment similar to the real HPC's, so you don't take up valuable compute time ironing out the kinks. It gives you 750 * 4 core A53 devices running at 1.2GHz, with slow interconnects and only 1GB RAM per node, buts costs less than $35k or so (my figures, not sure of final cost, assuming $35/Pi ex VAT plus costs for the Bitscope racks)

          2. James Hughes 1

            Re: Pi flavour?

            Also, you can buy The Pi Zero and Zero W in quantity, but they will cost more than the single unit price. I'd go with the Pi3, so much faster.

    2. fajensen Silver badge

      Re: Pi flavour?

      How do they buy these amounts? Amazon Mechanical Turks buying many, many one-off's, special access or parallel manufacturing?

      1. Old Used Programmer Silver badge

        Re: Pi flavour?

        It's trivially easy to buy most Pis in bulk and possible to do so for the two models that are not commonly available (Pi0 and Pi0W) that way. Just contact either Allied or Newark and tell them how many you want.

        1. James Hughes 1

          Re: Pi flavour?

          Raspberry Pi make what is it about 500k a month or something ridiculous so buying in quantities is not a problem.

          As for using something like a banana pie or an orange Pi yes they do have faster ethernet but the os support is absolutely rubbish. Since this is a educational project rather than a serious high-performance computing project it seems that the Ethernet is probably not that important anyway.

  6. CraPo


    ...but can it play Crysis? (seriously, how many FlOpS are we looking at?)

    1. Anonymous Coward
      Anonymous Coward

      Re: Obligatory...

      Yes but you need 3 data centres full non compromised by shit request for pornatars

    2. Anonymous Coward
      Anonymous Coward

      Re: Obligatory...

      > how many FlOpS

      says the Pi3 GPU provides "24 GFLOPS of general purpose computing performance", which would make a not-too-shabby 18TF for the cluster of 750, if you can code for the GPU.

      OTOH, it says the Pi-B did 0.065 GF on LINPACK single-precision, which is rather less impressive.

      A single NVidia Titan Xp claims >10TF by itself.

      1. James Hughes 1

        Re: Obligatory...

        The latest Pi version is a quad core arm A53 running at 1.2gig so considerably faster than the original Pi model. They also support neon in the latest models so that if you can take advantage of it give you a massive speed improvement.

        There's a thread on the Raspberry Pi forums that has a lot of linpack style testing done on it which might be worth a look if you're interested.

  7. Warm Braw Silver badge

    That PoE means some serious Ethernet cable...

    3W per blade * 150 units per blade * 5 blades in a cluster = 2250W. Or about 47A at 48V. Or around 20 standard ethernet cables using all 4 pairs at 600mA per pair (802.3bt).

    I'd have thought a 13A plug would be more appropriate - or one of those washing machine outlets, since it's the US.

    1. Anonymous Coward
      Anonymous Coward

      Re: That PoE means some serious Ethernet cable...

      I'd imagine that the internal blades are powered via PoE (since it also says there's internal switching), but that the outside power is indeed 110-240V.

    2. Jim Mitchell Silver badge

      Re: That PoE means some serious Ethernet cable...

      Washing machines use regular outlets. Electric clothes dryers require up-rated wiring and receptacles. Unless you mean the water for cooling... ?

      1. Anonymous Coward
        Anonymous Coward

        Re: That PoE means some serious Ethernet cable...

        The Raspberry Pi 3B get their power from the BitScope Blades.

        Each Blade Quattro mounts four Raspberry Pi. The Blade gets its power through mounting holes at either end.

        The Blades are attached to two power plates - positive on one end and ground on the other. In this way BitScope can mount up to 15 Blades in each power pack. Depending on the Blade used (Quattro or Duo) 60 or 30 Raspberry Pi can be mounted. There is no power cabling other than to the power plates.

        BitScope then use multiple power packs to build out a Module. In this case the node count is 150 - with integrated networking and power.

        It is a VERY inexpensive but massively parallel system for research, not just to buy but to operate as well.

        A typical 42U Rack will hold seven modules - 1050 nodes.

        The only cabling is for network connections and this is simply for networking - NOT POE.

      2. Gotno iShit Wantno iShit

        Re: That PoE means some serious Ethernet cable...

        Washing machines use regular outlets. Electric clothes dryers require up-rated wiring and receptacles

        Only if your regular outlets are shit. The US or Europe for example.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021