back to article How to build a BONKERS 7.5TB, 10GbE test lab for under £60,000

In part one of The Register's Build A Bonkers Test Lab feature, I showed you how to build a test lab on the cheap; great for a home or SMB setup, but what if we need to test 10GbE? Part two is a wander through my current test lab to see how I've managed to pull together enough testing capability to give enterprise-class …

COMMENTS

This topic is closed for new posts.
  1. This post has been deleted by a moderator

    1. Tom 7 Silver badge

      Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

      and there may be hidden charges for more than one core. Worth checking - I've seen what seems to be a £1500 'investment' in MS server rise to ten times that - that was a few years ago and the last time I used MS software.

      1. Anonymous Coward
        Anonymous Coward

        Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

        That would be if it was datacenter edition, that is then per socket. Standard and Enterprise are per server licensing with enterprise having licensing rights for 4 VMs on the same host (standard 1 VM, datacenter unlimited). Microsoft hasnt done per core licensing until SQL Server 2012 came out, and forcing core licensing for the Enterprise edition, making it extremely expensive now.

        Server 2008 may have been mentioned as it would be easier for most people to set up, and from what I can see its not an advice article, its a look at what I've done and how much it cost me article.

        My file server at home isnt windows, was at the start, move to opensolaris, then updated solaris express, as i wanted to use ZFS, with 20TB of storage, cost me over the years probably around £1000 to build.

        1. nick turner

          Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

          Actually the licensing model has changed with the release of 2012, I don't know if you can even buy legitimate licenses for 2008r2 any more, you certainly can't in the enterprise. Now for the commercial versions both standard and datacenter are licensed on a per core basis with both being 1 license per 2 cores. The virtualization model is different too with standard allowing 2 virtual instances per license and datacenter unlimited (both are linked to hardware.

          1. Anonymous Coward
            Anonymous Coward

            Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

            The licensing model for server is still socket / physical processor based, not core based. Just now its 2 sockets / physical processors for each license.

    2. vmistery

      Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

      Poor advice? I don't see your point - does it matter what OS it is using in this case? its a nice little experiment! It also shows the very real ability to create very decent performance at a very reasonable price whether you would use it just in test or in Production or just do for a bit of fun. I like recycling our old kit into testing machines as it is rare they need anything more than new disks. I too would not personally use Windows for my shared storage but this chap obviously wanted too (probably because learning to do this in another OS would cost him more in the long run).

      1. Roland6 Silver badge

        Re: does it matter what OS it is using in this case? Yes!

        If the purpose is to build a test lab which provides a realistic micro test environment for enterprise equipment then it is obvious that Windows Server 2003/2008 has to be in it. Yes it may be nice having the capability to dual boot etc etc, but as a first step using a license, that I assume is paid for and hence no annual subscription, makes a lot of sense.

        1. Anonymous Coward
          Anonymous Coward

          Re: does it matter what OS it is using in this case? Yes!

          "If the purpose is to build a test lab which provides a realistic micro test environment for enterprise equipment then it is obvious that Windows Server 2003/2008 has to be in it."

          So you're saying that MS's OSes are the only OSes suited to enterprise computing? Hmmmm... Intriguing view point.

        2. JEDIDIAH
          Linux

          Re: does it matter what OS it is using in this case? Yes!

          > If the purpose is to build a test lab which provides a realistic micro test environment for enterprise equipment then it is obvious that Windows Server 2003/2008 has to be in it.

          Hardly.

          Windows is for small shops that can't affort IT staff.

          If you are talking "enterprise", then some form of Unix is the obvious way to go.

          1. Chris Holt
            Flame

            Re: does it matter what OS it is using in this case? Yes!

            Contrary to the real world where that is at least an 80/20 split if not 95/5 split in favour of windows. Show me a large enterprise (>5000 users) that does completely without Windows and Office or at least achieves 50/50 or stop claiming that windows is only for small companies

    3. Anonymous Coward
      Anonymous Coward

      Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

      @Eadon - do give it a rest... A mention of how to do something with Windows is no more of an advert than a mention of how to do something on Linux is an advert for linux.

      If you need a supported version of Windows Server, yeah it'll be about $750, but then again a supported version or RHEL will be about the same.

      Now, if you've got a test rig and want to run Windows Server, you'll get several installs of each different version (plus a whole load more) for about £100 for a technet subscription.

      1. This post has been deleted by a moderator

        1. Anonymous Coward
          Anonymous Coward

          Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

          @Eadon - Yes, support is not the same a a licence, but if you want an equivilant level of support+licence with Linux you'll be paying the same for the level of support+licence for windows. Centos is great, but there is no official support, this was kind of my point, you can get Windows for only a few quid per seat if you use technet, yet it's not free, but it's only a trivial amount of money. That said I'm glad your feeding yourself now, it must be very difficult, what with all the trolling you've got to get on with.

      2. JEDIDIAH
        Linux

        Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

        ...except I don't need to use a "supported" version of RHEL.

        I can just use Debian.

        This a domain of computing where you don't have suffer from the "learned helplessness" of a desktop PC user.

      3. Anonymous Coward
        Facepalm

        Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

        Did he say he was not installing Lunix? Or that it's an extra cost to install Windows Server on a test system? IE, one your going to test multiple OS/s setups and systems on? Why mention the zero cost of things that don't cost anything?

    4. SImon Hobson Silver badge
      FAIL

      Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

      > Is Microsoft paying you?

      If you actually READ the article, you'd see that he does in fact mention open source/free software. Did you not see the bit that starts "Unless you rely a great deal on open source ..." ?

      Yes, for some of us, Linux (or other FOSS of choice) does the job just fine - in fact all the servers I manage are Linux. However, there are actually a lot of jobs where Windows is the norm, and frankly that's not going to massively change in the very near future. Lots of big organisations are wedded to MS stuff, well and truly in so deep that it would be very difficult to extricate themselves if they wanted to. For example, regardless of what you or I may think about them on a technical or ethical level, things like Active Directory and Exchange do "just work very nicely" for those who have a MS only setup (which is still most large businesses).

      As others have suggested, your style of outbursts are actually very unhelpful when it comes to "selling" non-MS into corporates. People at all levels see these sorts of outbursts and assume that it's representative of everyone involved with free/open source software - and it's enough to give them a bias against before you even open the conversation. Really, it's not helpful.

      So, assuming Trevor Pott is working for an outfit, or in industries, where MS use is the norm, then his test lab needs to run MS as well. Running Linux on the test setup would be 0% useful if the requirement is to test new setups of (say) clusters of servers running MS SQL Server.

      1. Trevor_Pott Gold badge

        @Simon Hobson

        My testlab will ultimately run Windows, VMware, Openstack, Cloudstack, various flavours of Linux and several BSD disties that I am playing with. Don't worry though, I'm totally a paid shill for [corporation_you_hate] as well as a religious freetard while simultainiously lacking any understanding of anything because I don't do what [commenter_1] says AND what [commenters2-n] say, despite the fact that they all completely disagree.

        I write things on the internet, getting on 3 years now. After a time, it's all déjà moo. The handles attached to the blocks of text may vary, but the level and stinkyness of the bullshit contained in that text does not.

        That said, the chief grand poohbah around here gave me a gold commenttard badge, and that is a great thing. It comes with an ignore button. The SNR has increased dramatically since I started using it.

    5. Anonymous Coward
      Anonymous Coward

      Why did the register publish this article?

      These kind of free advertorials bring in revenue from the tech companies, nothing wrong it that ...

    6. phuzz Silver badge
      Trollface

      Re: "Windows Server is $750" - Word to the wise - a Linux server does the job better for $0

      "a Linux server does the job better for $0 "

      Which is why no enterprise uses Windows of course...

  2. Ian 55

    It's the price of the network cards that's surprising me

    I remember when you had to pay that to get 10Mbs...

  3. K
    Thumb Up

    Supermicro...

    I've got the little brother of this (4 servers, aka Dell C6100) in my test lab, comes with 8xQC Xeon CPU's 128Gb RAM and 12x1TB HDD's, cost me just over £2k.

    They are great value :)

  4. Steve Foster

    2.5" bays for 5.25" holes.

    Since you like SuperMicro, I suggest:

    http://www.supermicro.nl/products/accessories/mobilerack/CSE-M28E2.cfm

    If you don't need the dual-port capability, the basic model is:

    http://www.supermicro.nl/products/accessories/mobilerack/CSE-M28E1.cfm

    4 bay version:

    http://www.supermicro.nl/products/accessories/mobilerack/CSE-M14.cfm

    (don't ask me why supermicro.com redirects me to .nl!)

    Budget range:

    http://intrl.startech.com/HDD/Mobile-Racks?filter_DRIVESIZE=2.5in&page=all

    1. Trevor_Pott Gold badge

      Re: 2.5" bays for 5.25" holes.

      Seriously; we had the Icy Docks on order at the local computer retailer. They were all set to arrive...then didn't. Oh, but they'll arrive! They'll be here in time for your review! Nope. Really should have bought from an etailier instead of waiting for the local retailer to get kit in, but ya can't win 'em all...

  5. The Alpha Klutz
    WTF?

    tens of thousands of dollars spent on MS licences - really?

    Could give the money to any semi-competent Linux grey beard and end up with something that won't be obsolete at the next Microsoft product cycle.

    1. Anonymous Coward
      Anonymous Coward

      Re: tens of thousands of dollars spent on MS licences - really?

      Yawn, how tedious and predictable.

      In other news Windows 2003 R1 still supported, a decade later.

      1. Anonymous Coward
        Anonymous Coward

        Re: tens of thousands of dollars spent on MS licences - really?

        "... still supported, a decade later." Not the Small Business edition. Making the outcome even, when inputs are not even is not called fair judgment or open minded by any stretch of the imagination. It only makes you feel like you are being fair or open minded especially when the majority are giving you that look that you don't know what you are talking about. So go on and keep being fair.

  6. The Alpha Klutz

    also

    if you can find any real company that is willing to throw money at this stuff, please, tell me where to send my CV. When my employers talk about test labs they mean the pile of broken pentium 2s.

    1. Anonymous Coward
      Coat

      Re: also

      Hahahaha. I feel for you though. My test lab is a pile of Pentium 4s, that I got for free, and it's only for home testing. A hobby. Not even paid. I've got a P2 in the cupboard, but I used that to test my drilling, hammering and Frisbee skills.

      Mine is the one with a multi tool in each pocket for taking apart the obsolete tech to see "how it works". Only after it's depreciated to a penny or two though.

  7. Longrod_von_Hugendong
    Holmes

    zOMFG....

    i stopped reading once it was mentioned using windoze, go and get a real OS, then we can talk.

    I cannot decide whether to use Fail, WTF or paris - think i will go with Sherlock, since i think some clue needs to be found...

  8. This post has been deleted by a moderator

    1. Anonymous Coward
      Anonymous Coward

      Re: Register for geeks? Biting the hand of IT or taking it up the ass?

      @Eadon: Your comments are just embarrassing, I have to use Linux professionally and have to spend a not inconsiderable amount of time explaining to potential users of Linux that people like you are not indicative of the people at the companies supplying Linux OSes.

      Newsflash: Lots of people use Non FOSS OSes and they need interop, not purist fanboys banging on about how they're doing it all wrong and stupid to boot. Do you think that your continuing banging on does your cause any good at all?

      1. This post has been deleted by a moderator

        1. Anonymous Coward
          Anonymous Coward

          Re: Register for geeks? Biting the hand of IT or taking it up the ass?

          So, it's ok for you to call the author of the article embarrassing, but as soon as someone does it to you, it's a personal attack.

          If you think windows is not compatible with anything, you're showing your true technical skills and I applaud that.

        2. BlueGreen

          Re: Register for geeks? Biting the hand of IT or taking it up the ass? @eadon

          I think the shill here is you. I think you shill by tarring linux users as unbending, warped and dislikeable. I hope MS pays you well, although not, I'm sure, enough to buy you the social skills which you faintly comprehend only by some painful hollowness within.

          > Everything else is compatible with MS Windows

          Yes. In Twattery, Veritas.

          So much has been ported to it and can be expected to run by running an installer. One of the many reasons "Micro$oft Windoze" (look, I did it for you) remains so attractive in large parts of the market.

          I'm trying to move to linux where my work permits. It really doesn't make some things easy though.

        3. Ronny Cook
          Facepalm

          Re: Register for geeks? Biting the hand of IT or taking it up the ass?

          "MS Windows is compatible with nothing. Everything else is compatible with MS Windows"

          CIFS was originally designed at IBM before MS hacked it into its current sorry state.

          Most Internet standards were developed externally to Microsoft. MP3, JPEG, many other standards originated outside of MS, although they do have a bad habit of adding non-standard extensions.

          The fact is, if you're writing an article aimed at an enterprise space, you need to recognise that there are people operating in that space who use Windows, for various reasons. I would love it if all our servers were running Linux, but that's not always an option.

          Windows does have some advantages. The file permissions model is vastly superior to the standard Linux 3-level model. Active DIrectory has some large advantages in managing a network. And senior management are so hung up on how easy it is to manage that they never notice that they need twice the people to do the same job. :-)

      2. Anonymous Coward
        Anonymous Coward

        Re: Register for geeks? Biting the hand of IT or taking it up the ass?

        Only reason why you need to support the dang MS ecosystem is because someone drank the koolaid. TBH, if I have the luxury of choosing to be creative and expressive in my work rather, I will do that. Yes, in this case, it does have to do with purism. The ulterior motives of commercial software companies really cramps creativity. If you don't create(highly likely in this day and age), you would not understand what I'm talking about.

    2. Greg D
      FAIL

      Re: Register for geeks? Biting the hand of IT or taking it up the ass?

      Windows OS is fine for server. In fact, it's easier to use and manage and there are many more management tools out there for it that make it a dream to use in a huge enterprise environment.

      If, like me, you look after a region of 8,000 users and 3 datacentres filled with servers, not mentioning all the virtualised stuff and project site servers, then Windows is a godsend. Easy to setup, easy to manage, easy to virtualise, generally works without much maintenance, full support is included in our EA (enterprise-wide license agreement), and its interop capabilities piss over Linux.

      Talk about fanboy rant!

      1. Anonymous Coward
        Anonymous Coward

        Re: Register for geeks? Biting the hand of IT or taking it up the ass?

        @Greg D,

        They could cut that down to two data centers if they dumped windows.

        Case in point, Microsoft can't even keep their reseller server going for them to get credit for sales.

  9. petur
    Happy

    how about simply buying a NAS?

    There are 10GBE boxes for quite acceptable prices these days... QNAP released some new ones just recently.

    1. Trevor_Pott Gold badge

      Re: how about simply buying a NAS?

      Show me the QNAP box that can FEED that 10GbE interface. None of them seem to be able to provide 1280 megaBYTES per second of storage movement...

  10. Lee Dowling Silver badge

    Just how many readers do you think have 60 grand to throw at a "test lab" that don't already know how to do the same thing cheaper and better?

    Honestly, Reg, that's the sort of thing I see on gamer forums with idiots throwing thousands of pounds at Microsoft in order to run a storage array. I don't expect it on here.

    I'd have been infinitely more interested in the article if it wasn't just "let's throw money at something that's not a problem if you have that kind of money", and even more if it wasn't focused solely on MS (which the writer claims he doesn't have the correct licensing for any further upgrade at the start - not surprising after the amount he's already wasted on it).

    Give us something "real world" that isn't aimed at people who could do a better job and write a better article than your own writers.

    1. diodesign (Written by Reg staff) Silver badge

      First-degree burns treatment

      "Just how many readers do you think have 60 grand to throw at a 'test lab' that don't already know how to do the same thing cheaper and better?"

      Just a couple of things. In case the headline didn't make it obvious enough, this is a bonkers setup: it's obviously an everything-including-the-kitchen-sink "dream rig", as the sub-headline states. Also, you don't have to buy the whole lot; the headline figure is tongue-in-cheek.

      We write technical stuff and people complain that's it boring; we write fun stuff and people call it fluff. You can't please all of the people all of the time.

      C.

      1. This post has been deleted by a moderator

        1. Anonymous Coward
          Anonymous Coward

          Re: First-degree burns treatment

          @Eadon - I know plenty command line jockeys who think that because they know some commands they somehow understand all computing, I prefer the mouse clickers because at least they're not sanctimonious about it.

          1. This post has been deleted by a moderator

            1. gromm

              Re: First-degree burns treatment

              This is about the best analogy I've seen for what GUIs do.

              My boss wants me to set up a GUI Dialplan designer for our Asterisk server so "our customers can modify their own dialplans". This is like saying that we should set up a GUI application designer so our customers can write their own software. What I've seen of the tools available is that they're *more* complicated than just building a dialplan by hand, and you have to already know what you're doing to begin with. If anything, it just reduces the number of typos in your dialplans. It certainly doesn't make the job simpler.

        2. Anonymous Coward
          Anonymous Coward

          Re: First-degree burns treatment

          I have to say, although I very much disagree with the tone used by poster "Eadon" in his comments, I would have to agree that a few *insightful* Linux articles would be welcome now and again.

          The general level even amongst those who write Linuxy stuff leaves a bit to be desired for my own taste. When I see one of those, I often head for the comments, as they tend to contain more interesting info than the article itself, which I suppose could be taken to prove that you already have a receptive audience to something a bit more technical.

          Just giving a bit of feedback, that is all.

      2. Anonymous Coward
        Thumb Up

        Re: First-degree burns treatment

        Thanks Diodesign for the article. It's incite full even for novices like me who cannot network 2 PCs successfully or setup a Linux Distro let alone Server 2003 or a Linux based one.

        I would ask though, does it run Crysis? Oh... I mean, is there a reason the test system cannot cheat with disk arrays? Do you need to test all the disk setups in the test bed? Or can you get away with "virtualising" that bit to save some cash? Setup half as many disks, but in software pretend it's a duplicate? Or max out a 10GB line by sending duplicate data from ram, instead of trying to feed it all from disk?

        I guess that is no help, as the test is if the disks can feed the ram/cache that can feed the network? So virtualising the disks means you could hit an unforeseen problem when switching to real hardware?

        1. Trevor_Pott Gold badge

          Re: First-degree burns treatment

          "switching to real hardware"...what?

          You mean you still run stuff that isn't virtualised in production?

  11. DS 1

    Hmm

    Maybe I am off here, but the problem with this - is its a bit of fantasy. Trevor seems to have access to a high level of funding that is beyond the norm. While it is fun to talk 10Gbps and 60K - the harsh reality is that this is all fantasy beyond reality for most of us. We are in the real world having to make and mend and build using very minimal costs. I suppose its nice to look in on this, but what fun is a project you can't actually do yourself, or partake in.

    Also, I have to say, MS and Windows are in terrible trouble. The OS licensing when in many cases you can do it for free is a real world challenge. What future for AD when you can't even hook up surface to it and group policy? What are you paying all these huge costs on licensing? For joined up cross device defined design? What with in tune and azure?

    1. Trevor_Pott Gold badge

      @DS1

      Yes, it is a lot fantasy. It is a Bonkers test lab, after all. But the stuff I detailed in Part 1 is realistic and achievable. The Kingston Hyper-X array should also be realistically achievable for most, if the "high speed storage" part of the equation appeals to you.

      The 10Gbit network with added WTF was built as much to see "can it be done" as anything. My hope is that having such a test lab lying around will allow me to do better reviews on more relevant equipment for The Register than I would otherwise be able to do.

      Do we want to limit ourselves to reviews of the latest iPhone or consumer home NAS? Or do we want to occasionally tear apart some bit of midsize gear or even enterprise kit? If we do want to be able to throw that more powerful equipment on the bench and give it a run for its money, someone is going to have to build a bonkers test lab. So I did.

      1. Anonymous Coward
        Anonymous Coward

        Re: @DS1

        While out of "most of our pockets", 60K is nothing to a business AFAIK. I've seen people sink more pocket money into a house to buy/sell/rent project. If someone wants to make a server/system/service and it's not property, but internet based, well hopefully this article lets them know it's possible.

        Now... where to get funding for my Bonkers Server business for downloading cats eating burgers website... ;)

  12. thondwe
    WTF?

    OTT

    Technet not enough to cover this lot (instead of spend $$$$ on Windows Licences)? Where does Technet run out of steam? To setup a test bed for AD/Echange/System Centre 2012 you'd be spinning up a fair few VMs - so where's the problem? Especially if you use the free HyperV for the physical boxes.

    If you need to licence it properly, it's not really a test bed is it? Better to build a smaller local test bed and run the rest on Azure (some free time for that is included in an MSDN licence!)

    More money than sense this man!

    Paul

    1. Trevor_Pott Gold badge

      Re: OTT

      It's a testbed that needs licensing the instant I have to maintain "test" (or as we often refer to them "sandbox") copies of running instances. For example, my largest client has 250 VMs in production, among them there are 23 different "classes" of VMs. Each of these classes needs to exist in my testlab environment so that I can do things like test patches, the latest version upgrades to software and more.

      In fact, this testlab just received its last components in the mail last night and they have already been pressed into service. That said, I use a single datacenter license to achieve this, and the rest of my lab runs Linux, as this is now the bulk of what I have deployed, and thus the bulk of what I have to test.

      As for running the rest on Azure: no. For one thing, the cost of storage is too much, and my test labs often require the ability to access a significant subset of the live data for testing. For another, the laws of my nation do not allow me to store personally identifiable information on countries without robust civil liberties and privacy protections. That means the US is out, and trans-Atlantic data flinging in order to store in the EU is expensive.

      I'll build my own private "cloud" thanks, and run my testlab requirements – and those I need to test the builds my clients have – on it. It's far, far cheaper over the expected 6 year life of this equipment.

      1. Anonymous Coward
        Joke

        Re: OTT

        I was with you Trevor until you said "cloud*"... oh wait, you used quotation marks too. all forgive. :)

        *internet/server/online/buzzword

  13. Anonymous Coward
    Anonymous Coward

    SAN is dead

    "up to 32 gigs of RAM; the maximum currently supported by VMware's free hypervisor." - Use KVM on RHEL or Ubuntu - it scales better. If you want to pay for support, use RHEV and buy commonly available, enterprise grade x86 hardware.

    A RAID card is not necessary, software RAID is all you need.

    "Unfortunately moving virtual machines from node to node in this configuration is slow and frustrating." - Trevor, you show the example of using 10GbE, you can set up Etherchannel and/or you can use Gluster to have additional resilience - that's the beauty of OSV. See the advantage of a Linux/DAS storage grid, the data is clostest to where it is needed and you get resilience built in - not possible in a in a traditional SAN setup unless you spend a pile of cash.

    "Being a recycled server" - you hit it on the head there, using off the shelf hardware you can easily take one or more storage nodes out of the grid, replace motherboard/PCI cards/disks and put it back in again. This reduces TCO and allows you to plan your storage requirements more granular.

    http://www.linux-kvm.org/page/How_To_Migrate_From_Vmware_To_KVM

    http://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.kvm.limits.html

    1. Trevor_Pott Gold badge

      Re: SAN is dead

      Gluster is on my list for later in the year. And RAID cards have some distinct advantages over software RAID. Specifically when you start pushing 1000 megaBYTES per second or higher through them. Software RAID is fine if you RAIN. It isn't so fine if you only have the equipment to build a single, reliable and eye-bleedingly fast storage node.

      *shrug*

  14. Jeff 11

    Trevor, did you look at using Infiniband instead of 10GbE? Obviously it wouldn't be suited to a production environment where you already have the switching infrastructure in place, but I'd be interested to see if you could shave £10k off the price for ostensibly the same set of capabilities...

    1. Trevor_Pott Gold badge

      @Jeff11

      I did; but I couldn't get anyone to send me Infiniband gear, nor are there local suppliers that offer it cheap. So if I did invest, it would a) be stupid expensive and b) I would be in deep ca-ca-poo-poo if anything whet splor and I needed a spare ASAP.

      I call this my "test lab," but I should point out that my "live" corprate VMs occupy 1/5th of this particular setup at any given time. (Actually, they fit just fine in a single Eris 1 node, but that's a whole other story...)

      I would love to test, review and otherwise learn about infiniband. With luck, some will show up on my doorstep one day.

  15. Bob H

    Would be nice if El Reg did a similar article for the cheap home made infiniband networking that I have seen, there are some good on-line howtos already, but would be nice to see a reviewer do one.

    1. Trevor_Pott Gold badge

      There has been a lot of questions about this. I have put it on my list, I promise you. :)

  16. Anonymous Coward
    Anonymous Coward

    Bypassing the usual flamewars...

    I'll just ask - what on earth is that apparently several thousand $'s worth of kit stacked on top of ? Looks like one little bump could end your test lab in one go.

    1. Trevor_Pott Gold badge

      Re: Bypassing the usual flamewars...

      Are you asking about the UPS from the dark ages, or the terrible IBM rack? Or the built-like-a-tank-will-never-ever-ever-ever-ever-die orange chair of doom?

      Don't question the chair. The chair is indestructible. (And it has an equipment seatbelt.)

      For the record though, we spent all night racking stuff last night...

  17. h3

    There is no point in getting i5's ever for this sort of stuff when you can get Xeon E3's so cheap and use unregistered ECC with them.

  18. ISYS
    Holmes

    Which OS to use?

    Suerly if the end game is to build a test lab, then you will need to install an OS appropriate to what you are testing. Depending on what you are testing, this could be all Linux (various flavours), all Windows or in most cases a mixture. In my experience (20+ years) - a corporate network is Windows based, webfarms are Linux, Databases are MSSQL if you are using relatively small amounts of data and Oracle if you are using more.

    Linux is also the weapon of choice when it comes to firewalls and proxys.

    Like somebody has already mentioned - WIndows is used in corporate environments because it is easy to install, use and maintain. There are a lot of people out there who know it and therefore it is relatively cheap for compainies to hire people to maintain it.

  19. unixguy
    Linux

    64 cores 512GB of Main Memory and KVM

    Imagine 64 opteron cores and 512GB of memory creating almost any VM guest imaginable for the cost of the hardware only. This is the modern day reality of a Dell R815 and Centos 6.3. I will not argue the religion of what operating system is better for this or that. I will share the most cost effective way we have found to implement virtual guests, and then we can talk about guest management hardened externally facing systems (ie. administration from the virtual machine console and no remote access to the guest over the network) Now we are starting to talk enterprise class!!

    1. This post has been deleted by a moderator

  20. Nate Amsden

    brocade

    another option for cheap 10GbE would be the Brocade 6450-24 which is a 24 port 10/100/1000 with optional 4 10G SFP (I think there is a software license requirement here). The base switch seems to run under $2k at the low end, and has basic layer 3 abilities, the license add-on is in the ~$700 range seems like(2 ports). I'm not sure if the switch includes 2 ports of 10Gb licensing or none.

    I haven't personally used them though a co-worker has several deployed for our internal corp IT network, he has no complaints.

    for my needs juniper is far too complicated to work with, same for cisco (brocade is similar to cisco). But I suppose if your just configuring it once and not touching it after that then it's not too terrible.

  21. Anonymous Coward
    Anonymous Coward

    Eadon I understood what you were saying

    Eadon I understood what you were saying and agree

    Yes it is a very nice test lab if you happen to have surplus kit lying around / you have some arrangement with the suppliers or you have some company profits that need to disappear.

    If I was getting it all for free I wouldnt kick it out of bed but then again I wouldnt spend money on this setup when there are better solutions that are cheaper.

  22. This post has been deleted by a moderator

  23. Rainer
    Go

    Interesting article

    The headline is a bit misleading as the hardware alone is much cheaper.

    I'd love to see how several different on-premise cloud solutions perform on this kind of setup, notably:

    - openstack

    - proxmox

    - SmartOS with the "cloud" GUI from this guy: http://blog.smartcore.net.au

    Certainly not a #FAIL article. True, I'm not the least interested in how well HyperV runs on that - but that does not diminish the value of the article or the information contained within it.

    1. Trevor_Pott Gold badge

      Re: Interesting article

      It's amazing how often commenters get bent out of shape by a title, instead of the comment. (Or by two paragraphs of an article, ignoring the entire rest of it.) *shrug*

      That said...I now have sexy testbed. I have requests from folks to test openstack and cloudstack. I already have plans to test Hyper-V and VMware. I will add your recommendations of Proxmox and SmartOS to me list. What's the point of putting such a lab together if I can't test the things on it that matter to our readers?

      The Fat Twin arrived. I was expecting it to come with a variety of configurations, apparently that didn't quite happen. Instead, I have 4 identical nodes: 2x Xeon E5 2680 /w 128GB RAM and 2x 480GB SSD. Should be good enough to give any of the virty stacks a run for thier money, no?

      When the petty cash refills, I'll fill the other 4 nodes.

  24. Anonymous Coward
    Anonymous Coward

    About the clicky admin articles

    It's true. Every build/lab/howto I've read on the Reg is useless to me. I don't want to belittle those writing, but honestly, it does sound like a bit of filler. I mean, I could write you an article about how to set up the jack under my car and it might contain more useful tips that this. The best part about it was pointing out the switch. I'm looking for good 10GBASE-T hardware. Perhaps do a review of that. I'd read that.

    1. Trevor_Pott Gold badge

      Re: About the clicky admin articles

      As soon as I get good 10Gbase-T hardware, I'll review it. I should point out that a review of the Supermicro and Dell switches is coming up here soon (i am just putting it in to the CMS now) and that the Dell switch in question does have a 10Gbase-T variant. (Albeit slightly more expensive.)

      That said, if and when you have requests for things to review/do a how-to on etc...ask! I am (naturally) limited by what I can get my hands on...but I've been working hard to build a lab that will allow me the flexibility to do reviews on damned near anything. Maybe I can meet the request, maybe I can't...but I promise you, if readers ask for it, I'll do my level best to get hold of it and put it to the test.

      You can also help by providing suggestions as to what tests you would like to see run. Contrary to popular opinion – especially those of the berate, denigrate and wail like spoilt chillum crowd – I do this "reviewing products" thing mostly to try to help. Not every article will be thought provoking or insightful to the totality of the readership, but I do hope that each one provides some benefit to at least some of them.

      In the meantime, I'll poke some 10Gbase-T vendors and see if any are willing to have their switchen wrung.

  25. JEDIDIAH
    Thumb Down

    ...and now for something completely different.

    I dunno. Blowing a lot of money on something outlandish just doesn't seem that interesting. On the other hand, it is pretty trivial to max out the more common tech. All it takes is a single spindle really. What would be more interesting is seeing how easy it would be to "take it up a notch". How accessable is another level of performance above and beyond what's cheap and readily available.

    There seems to be a big gap here and the interesting story I think is how you could make that gap smaller. My own SSD-free setup could probably benefit greatly from such an incremental improvement.

  26. Anonymous Coward
    Thumb Up

    "cardboard and duct tape"

    I have 2 2.5inch laptop drives mounted with lego in my rig. One is a spare I had so used for backup, one is "about to die", so I'm testing it to see when it dies and using it as a kind of scratch disk/test disk for now. I don't mind if I trash the broken disk, so it's a good one to experiment on. :D

  27. Chris_B

    Re: First-degree burns treatment

    I think it might be time for Eadon to put his money where his mouth is a write "that" article that he thinks is so missing from ElReg.

  28. IHateWearingATie
    Go

    How many commentards didn't read the words 'Test Lab'?

    I'm not a hardware techie, but it would seem obvious to me that, unless all of the IT shops that you deal with are 100% non-windows, your TEST LAB would need Windows in it at some point.

    After all, TESTING would seem to be the point of a TEST LAB., whether or not you agree with the decision to use Windows or not.

    Keep up the articles like this - while my professional sphere will never get this techy, its an interesting read all the same.

  29. Anonymous Coward
    Anonymous Coward

    Ah, the test lab...

    For the large part, our test lab is a strange mix of recycled kit and the odd new bit here and there. We've an old poweredge R805 or two that are out of warranty doing the grunt work of running the virtual servers in the lab. For storage, they back onto a Netapp FAS2050 that used to be in our DR bunker. for one of the line of business apps, we have a pair of wheezing old IBM power5 based AIX boxen, and some associated support hardware. One of our plans is to eventually bring the various business apps up in there so that we can have a tiny, sandboxed version of the company's network that vendors can play around with and break instead of the production environment. :)

    Anon to protect my paycheck.

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2021