Re: Recovery redundant?
It depends on the nature of the servers.
If you have large numbers of similar or the same servers (well, built from the same image, with the same installed software), the rebuilding from sources or a gold image is quite possible.
If you have significant numbers of servers that have a long history, with different programs that may not necessarily be amenable to scripted install, large amounts of customisations, patches and upgrades etc, then rebuilding from source is not an option, at least not without keeping more people in your support teams than your management is prepared to afford.
Where I currently work, we have systems with over 10 years of history (the systems have been migrated to newer hardware at least once, and it's not Wintel, before you ask), and it would not be possible to 'rebuild from source' in a convenient time. The recovery process involves taking OS system images on a regular basis, and having a mechanism to restore these to the same, or different hardware, and then restore the data.
From long experience in this business, I know that automated install is desirable, and I've worked on more than one attempt on implementing, but without significant resourcing, it will and does break down over time whenever it becomes easier to do something by hand as a one-off, rather than adding to the installation process. And eventually, you have to start from scratch, and hope that the 'new' method results in an environment that functions the same as the old one.
But then again, the OS I work with is normally regarded as being less vulnerable to attack than some.
Biting the hand that feeds IT
