US boffin builds 32-way Raspberry Pi cluster Boise University PhD candidate Joshua Kiepert has built a 32-way Beowulf cluster from Raspberry Pis. Kiepert says his research focuses on “developing a novel data sharing system for wireless sensor networks to facilitate in-network collaborative processing of sensor data.” To study that field Kipert figured he would need a …
Is it just me that thinks that 4 fans is overkill for a device which basically doesn't need cooling in normal operation
The nice thing about having 4 of them is that you can run them at greatly reduced speed and still have enough airflow to matter, just make less noise doing so.
I believe the rack is open front & back, so all he really needed to do was to mount the thing in an angle (say 45 degrees) and there would be enough natural convection to keep things cool. However, it wouldn't look half as cool and you'd have a larger footprint - another advantage of his approach is that the whole thing isn't much bigger than a normal PC case.
Thumbs up from me.
You have to cool them because you have piled them into a rack in close proximity.
Now, if you suspended them from the ceiling by ethernet cables of varying lengths you would have a combination passively cooled Beowolf cluster and hi-tech mobile. A really practical and useful addition to office decor.
You may be on to something. ISTR that there were moves afoot in the Pi world for the things to support power over ethernet at some point in the future, which would make this eminently practical as well as outrageously cool.
You'd have the only cluster in the world where the failure of one of the little plastic tang things on an ethernet cable causes it to drop a node..........both figuratively and literally.
Spent $2000 on RPi's. Wasted $50 on absolute junk (LED's, unnecessarily flashy fans, etc.). Then taped it together with electrical tape. Spent a huge amount on a managed switch, which I can't see being used on a "all-machines-the-same" internal network (probably just a few 8-port switches with Gb port would have been better, actually)
Got performance basically in the range of, say, an 8-core x86 processor. For $2000 he could have just bought direct from here:
http://www.titanuscomputers.com/Quad-AMD-Opteron-up-to-64-Cores-HPC-Workstation-Computer-s/26.htm
And wiped the floor with it on a single machine, without electrical, cabling, machines, etc. concerns (and got a GPU for free, 8Gb RAM, stupendous storage not on SD cards, etc.). And his system takes 200W-ish, so you don't actually use much more power (the above would complete the task in less time, hence less power, and you could also use it as your main desktop so you don't have to recompile everything and chuck it over to the RPi's for computation from your laptop anyway).
I'm sure it's all a nice experiment and good "experience" for a quick play / setup with MP systems but it's really nothing interesting - especially not from a PhD candidate. Hell, I'd be rather peeved at him wasting his time and money on building that system and having the gall to write it up compared to buying a computer that runs his dissertation work directly without having to faff about. Especially the "performance" section of his write-up that successfully manages to imply that his system is actually worse than the others (performance per dollar) but with graphs that try to convey the opposite and then ploughs on to describe how much faster having more RPi's is than having just one (without any comparison to the alternatives).
If a 15-year-old had done this, I'd be saying good on them and well done for experimenting. But a PhD? Really? This is children's toys and the reason the "real machines" he wanted to use are booked up and expensive is because they wipe the floor with this for much less overall cost. Even the term Beowulf cluster died out many years ago when people realised that, actually, joining lots of commodity machines together wouldn't really beat the performance of whatever-the-next-most-expensive processor was, and if it did, then not by much and only for highly-parallel workloads. And nowadays, you find that the average desktop GPU will wipe the floor with even such a system (unless you have a cluster of GPU's of course, but that's an actually *interesting* project even if it's still old hat).
Way to completely miss the point. This wasn't built in order to provide a lot of processing power for the price. It was done as an economical way to provide a platform on which the PhD student could develop massively parallel software without having to share access to the university's "real machines". In case you still don't get it, this was NOT his PhD project, it was something he built to help him with his PhD project.
Like the other guy said. The problem is that this is a project not for adults but for novices and children. It's a proof of concept system at best. In that regard, it's very much like any other cluster built out of ancient or woefully underpowered hardware. It's the butt of every joke about building a Beowulf cluster of something.
The point of this cluster is not to see how fast it can go, but to work with a real (though slow) cluster. Clustering has its own issues, e.g. where data is, how do you make sure the right data is in the right place at the right time. Simulating a cluster on a big single CPU box is not the same as doing it for real.
There are lots of useful lessons to be learnt from doing this, perhaps one big one is that people incorrectly think a cluster is simply one big computer, its not.
For $2000 he could have just bought direct…
This is a custom build with lots of unnecessary duplication for a particular requirement.
I imagine any systems builder would be able to provide a rack-based system using a similar configuration for a fraction of the cost. Of course, they'd want to increase chip & core density to make it commercial. But being able to build systems at this price makes it a lot easier to try things out.
Guess the performance will compare pretty nicely once your Xeon PC is running 32 VMs so that it looks like a Beowulf cluster. There's a lot of point-missing going on in this comment thread; this is not built for throughput, but to work like a Beowulf cluster. The same goes for Glasgow's Raspberry Pi cloud - they're not planning to replace Google's data center anytime soon, just to understand and model it so they can develop for something that 'shape' without tying up the big one that does the real work.
Currently you can only use that tiny little ARM core on the Raspberry Pi, while the main processor, a large DSP used for it's video outputs, is largely unusable running some closed realtime OS.
Opening up that DSP would enable a whole new set of applications. For example you could do fast data processing, for example to do sonar or even radar on that little board.
No, not really. The GPU runs at 250Mhzx, and is a twin core 16 way vector/scaler unit (not a DSP). It's not hugely faster, and is ONLY faster when you can SIMD your code. Which is not easy in itself, and harder since it's all done in assembler.
The realtime OS on the GPU is threadx btw.
Opening it up wouldn't get the benefits many seem to think it would. There would be some, but not much for the majority of users.
"Who are we going to bait then?"
Fair point, but I'd say point at and laugh rather than bait. For Eadon all the bait that's required is "Microsoft" or "Windows" in the headline. Then he's off before most people have even read said headline. He's such a card :) He sounds fairly sane in his posts on space exploration stuff, so the Windows stuff is evidence of some major OCD thing, I'd guess. Sometimes I can picture him sat in front of the screen and splattering it with blood when the throbbing vein in his forehead goes pop ...
I used to be like that too. I grew up on little micros, first met MS properly when I got my first PC, and gradually fell out of love with them between DOS 5.0 and Windows 95. After that I was borged by the penguin army and moving from the (basically functional, but) irritatingly crude platforms I'd been used to over to this rather beautiful Unix derivative, combined with very strong opinions and over-confidence in my knowledge (I was a teenager after all) made me quite the zealot for many years. It took half a decade of coding for Windows to convince me that it's actually a plucky little platform and even its quirks are quite endearing when you've seen it soldier on, day in and day out, for years, doing useful work with respectable consistency.
So, as a former MS hater myself, I think it might have to do with an overestimate of the extent to which other platforms' advantages over MS products are actually *news* to anyone at this point, combined with a refusal to recognise that Microsoft actually do produce usable software. I still run Linux on my own boxen, with only one willing to occasionally boot Windows 7 for Kinect development; I still consistently prefer Linux when it's an option but no longer feel the need to ram it down everyones' throats to quite the same extent.
> It took half a decade of coding for Windows to convince me that it's actually a plucky little platform and even its quirks are quite endearing when you've seen it soldier on, day in and day out, for years, doing useful work with respectable consistency.
Soldier on? Consistency?
You've got to be kidding.
Microsoft lowering the bar with Windows is a very big part of why ARM devices are so trendy right now.
Yup, I'm only reporting what I saw. Several tens of Windows XP machines (mostly aging Dell desktops) running critical systems (the older stuff mostly written in VB6, the younger stuff mostly in C#). They were shielded from most threats by running few applications (mostly, one in-house application) and not having full access to the network (and no access to the internet), never running a browser and so on, but soldier on they did, surviving patches apart from a couple (I exaggerate not, I remember 2 occasions in 5 years across multiple versions where an OS or Antivirus update floored the machines briefly), and better management of the roll-out would have avoided those minor hiccups too.
My own development laptop was a Windows XP machine and never once fell prey to malware in the 3 years of general bashing it took (including plenty of temporary software installations and removals, which left a little cruft as we've come to expect but didn't break anything) and ran absolutely consistently (a few little niggles, nothing serious) until it was replaced by a faster model (also on XP) which performed similarly well every day until I left the job. Obviously as a 'tech savvy' user I wasn't taking risks with it, but I wasn't treating it all that kindly either since I knew I could re-image it in half an hour if necessary. It never was necessary though.
I know MS software has some weaknesses; all the Windows boxes were rebooted hilariously frequently from a Linux fan's perspective, and there were a million little silly faults (the file copy dialog's pointless animation stayed squint for *YEARS* between NT4 and Vista, where it was finally replaced IIRC) and the overall user experience is really quite bitty and unaesthetic in my view, but these are not serious problems. The machines did their actual jobs flawlessly, and the development tools (even lowly old VB6) were great.
I'm still a GNU/Linux fan myself, as I say, and prefer it when I have the option. Usually, when I use Windows, it's for a bad reason (mostly platform lock-in, the Kinect being a case in point), and I don't like the way MS license their software or behave towards other businesses, but I'm not going to pretend the flaws are worse than they are and I no longer gurn when someone wants something built for Windows. Fair enough, it's a perfectly decent target for development and will run what I write just fine.
... Just to be clear, if I'd been asked what platform these systems really ought to run on, I'd have recommended GNU/Linux. Any of my former colleagues would confirm that I raised the possibility at every excuse, ad nauseum. The machines were more powerful (and power thirsty, and physically fragile) than they could have been running Linux and disabling virtually everything on the Windows boxes meant we were barely running any of the code we were (as far as I was concerned, needlessly) paying for licenses to use. It's not the way I would have designed the system if it had been up to me.
Integration with other aspects of the technical environment would have been prohibitively difficult. We needed to talk to old COM drivers which would have needed to have been replaced or wrapped somehow, and it just wasn't worth it given that the Windows boxes worked, and the infrastructure people liked the remote management and monitoring features they had. The development teams all had Win32 experience, with relatively few of us having done much with Linux. Overall, for that business, Windows made sense for the bulk of what they did. (There were a few Linux boxes in the mix, too, and more were arriving around the time I left).
I don't think there's any sense being dogmatic.
Glasgow Uni has a Raspberry Pi cloud also:
http://raspberrypicloud.wordpress.com/
And they used Lego! I wasn't involved in the project (hence the apostrophes around 'us' in the title - just trying to give the impression I'm a real part of this institution, rather than a 31 year old undergrad desperately trying to hang on for another year, heh) but saw a couple of presentations about it recently as all the level four projects were winding up.
This isn't a Beowulf cluster of course, but it's another 'big' stack of tasty Pi so I thought I'd mention it.
I can't help thinking he would have been better off with a Mac Mini i7 upgraded to 16GB and SSD, and simply run 32 VMs on it for the sensor simulation. Smaller, more powerful, easily reconfigured, and as cheap or cheaper.
As for those talking about running Windows on it: this is 32 separate and independent machines each with 512MB RAM, not a 32 core SMP box where all cores have shared access to the same memory space. You'll need 32 licences plus some cluster layer on top to do anything useful (htcondor perhaps)
That was the only thing I was thinking, why not build an ok pc and put 32 linux vms on it same concept still pretty cheap if you only went with like 128MB per vm you still only need 4GB of Ram sure disk i/o would be an issue but as he said its not fast on the pi cluster either. Neat idea and all but i guess the cost conciousness in me says use a spare pc with ESXi or KVM and do the same thing without spending anything most likely.
Although it's not super-clear from the article, it's specifically the low-level IO that he was trying to talk to with the cluster, which of course you would have to specifically emulate with the hypervisor (as the point of the hypervisor is to present an agnostic interface).
Of course the other reason to do it (just as valid in my opinion) is "because".
This post has been deleted by its author
One can only hope that this new generation of ultra-low-cost computers will keep resulting in the design and construction of more and more massively-parallel (MP) computers.
Why? Because what is and has been missing from the MP scene in order to realize the full potential of MP computers is the SOFTWARE to fully make use of the hardware. As the cost of the hardware approaches zero--relatively speaking--we self-important computer engineers are going to have to stop kidding ourselves that we are doing something REALLY IMPORTANT by designing and building the MP machinery; we've got to finally 'fess up to the fact that all along, that's been the easy part. Now we've got do the really HARD work: design the assemblers, compilers, and high-level languages to make it all really, REALLY WORK.
And one more nice feature of low, low cost MP computers: don't bet against a ten- or twelve-year old boy OR girl--or a group of them--doing the seminal work on the MP critical software (their advantage over us? They, unlike us, don't know that it can't be done).
Regards...
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
