Reply to post: Re: This smells like BS.

Sysadmin trained his offshore replacements, sat back, watched ex-employer's world burn

Anonymous Coward
Anonymous Coward

Re: This smells like BS.

Hahahaha cough.

If you've ever had the dubious pleasure of using Bladelogic, particularly when implemented in a very large supplier of managed hosting services, you'll know what shouldn't be possible sadly is.

One poor admin working for a single client followed the documented script to execute a job on a portion of his client's estate. The script was a battle-hardened "sudo killall java; shutdown -r now" designed to force everything down quickly, and written in nsh to work across Windows and *nix.

Unfortunately Bladelogic didn't highlight what you've selected in the tree - in this case "Global" rather than "Client X, Datacenter 1", and in a cost-cutting American tradition nobody had bothered to configure it to restrict admins to only be able to effect change on their permitted systems.

He realised quickly, but not before c.200k machines went kerplunk. Total service penalty costs in the region of 8 figures I believe.

AC obviously.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon