Data centres: what are the new skill sets?

The capabilities of modern servers offer in-principle benefits of more dynamic management, workload balancing and so on. How do these capabilities impact on the skillsets required of data centre operations staff today? Trevor Pott Trevor Pott Infrastructure Support Engineer The commoditisation of certain technologies once …

[Sir Runcible Spoon]

re:Jack of all trades

Whilst it's true that there aren't as many positions for jack of all trades operators (thus most people seem to specialise) it does leave a small, but workably signifcant market for people who specialise in being able to do a bit of everything.

Perhaps not employed in the actual field of their knowledge, but a general background of all things technical, coupled with a good range of 'soft' skills can make all the difference when contracting. It's still difficult to secure a position, but anyone who can organise and relate to various parts of the business and have a good understanding of budgetry requirements, time constraints and internal politics (and how to avoid them) will more than likely get the renewals that make contracting viable.

This is the path I have tried to take and I'm still only on my fourth contract in 8 years, being fully employed throughout (apart from the 4 month 'holiday' I took this summer :)

[Anonymous Coward]

lots of parallels

My old school physics teacher complains that he cannot understand his new Honda - even though he taught auto engineering.

Same with the data center, how on earth do you know what is happening with hundreds of apps interacting on top of an dynamic, ever changing infrastructure. Talk about a house built on sand. Virtualisation/elastic environment = sharp knife = blood on floor.

Complexity has forced the auto manufacturers produce analysis boxes to pinpoint problems in their vehicles. I need one for my e2e processes but unfortunately, we are not there yet and there is little on the horizon.

Come back squawk box, all is forgiven.

[Nate Amsden]

ssh, bash perl?

If your seriously using ssh bash and perl alone to manage large numbers of systems, even smallish numbers(more than 50) then you really should look at better tools like cfengine or puppet.

ssh bash and perl alone are far too limited to handle real distributed admin work, unless of course your writing something along the lines of cfengine or puppet in perl ..

Just last week as an example my company built a new edge location in London(I'm in Seattle). We sent one guy out with 1 server running ESX and a VM with enough data to seed the rest of the network. He configured all of the remote management cards, the network guy got the VPN up, and I remotely installed ~55 physical and virtual machines in roughly 6.5 hours over the WAN including all application configuration and everything. And that was using real remote installs, not stupid image-based crap.

It could of been done faster though I decided to throw a wrench into things and revamp my kickstart setup at the same time which made me have to go troubleshoot some things and re-install some systems to fix them.

But all in all it was a pretty good day.

Wouldn't of been possible in that time line to do it without cfengine though(I haven't used puppet hear it's pretty good). My cfengine configuration is about 20,000 lines, to give an idea of the number of variations and tests that the system runs to ensure compliance, and that compliance is re-checked every single hour of the day.

I suppose I consider myself a jack of all trades guy whether it's systems, storage, networking, automation, monitoring, and architecture. It's really rare that I physically touch hardware these days.

I don't personally know anyone else that has my skill sets, though they weren't easy to come across, made a lot of sacrifices over the past decade(well worth it in the long run I believe, no regrets).

Pick the right products and you can scale pretty damn well.

[Trevor Pott o_O]

@Sir Runcible Spoon

Oh, no denying there's still (currently) a market for us old-timers. Jack of all trades admins are still hugely relevant in the SME space.

There will always be a limited need for jack of all trades types...but I fear the number of us that are required is shrinking. I am sure it’s cyclical…all IT job patterns have seemed so far to be, however what seems to be happening is a higher demand for narrowly-focused, (and most importantly cheap) specialists.

Smaller organisations are more and more outsourcing their IT. Those outsourcing companies, be they Dell, HP, or your local consultancy are slowly evolving along the same path. "Customer support staff" at the edges calming the customer down and finding out the problem, passing the information back to their actual "doers" who then push the button to fix the issue remotely. In the case of a dead bit, the backup is activated, and the customer sends it back to the outsource company for replacement. (Commodity hardware has gotten to the point that this is easily doable.) It is all very process-managed and sterile.

I don’t claim that this is the best way to do things…but simply that this is the direction that our industry is heading. IT is largely no longer about innovating or solving a problem with a unique solution. Almost everything you could want comes as a pre-packaged appliance now, or can be solved with easy-to-manage nifty virtualisation tricks.

Like the handyman of the physical world, the IT generalist will probably be the only of the "IT trades" that a company keeps on staff. The handyman can fix your doorknob, change the lightbulb or coax that creaky boiler back to life. If he needs something done outside his particular expertise, he calls up a plumber to have new pipe run, or an electrician to wire a room to code. We’ll be calling up a cable monkey to run lines, a datacenter specialist to install rack/power/cooling or a programmer to add functionality to a website.

Pay attention to how many handymen even large companies employ…and what those fellows get paid. They are what IT generalists are set to become. (On a totally selfish note however, it gives me no end of glee to picture several of the "specialists" I know as the digital plumbers of the future. Sadly, the "IT Specialist" types in my neck of the woods really look down on us generalists quite a bit.)

Good luck to us all!