Reply to post: Re: 9.5 hrs of downtime

AWS power failure in US-EAST-1 region killed some hardware and instances

spireite Silver badge
Holmes

Re: 9.5 hrs of downtime

Good old DR planning,

Back in the early 2000s, one of my old employers had such a thing, lasted about 4 years. All server kit WAS in the office.

In late Feb/early March every year, we'd troop off to a facility with backups retrieved from an offsite storage company (IronMountain).

Our kit at the time was big black cubed IBM NetFinitys.

What was priority? Email.... we were essentially a data company, but email was priority (Lotus notes).

Other systems were SQL databases of a couple of vendors, SSRS.

Of course these were bare metal restores, and the bare metal was something like HP DLxxx, or Dell 15xx. Couldn't find NefFinities for love nor money

We'd be getting ourselves to business critical up and running in three days.

In fact it proved a few points.....

1. Recovery of email is NOT critical

2. Stick to hardware that is more commonly available - not NetIndefinitely

3. No matter how you document, or how thorough you are. It will go wrong

4. Ensure your dev work is properly source controlled. Ours was in VSS. That s**t isn't stable on a production machine, you're doomed on a bare metalk restore frankly.

5. Upper management cannot prioritise for toffee.

6. Testing becomes "Can I connect, can I select a row etc".. Full test suite?? Yeah right... "We can't afford the man time for that, they have jobs to do in HO"

and

6. In a DR centre noone hears you scream. (good, they'd think a zombie horde was being fought)

My takeaway ? On paper a good thing, in reality barely above pointless.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon