Reply to post: If you're not testing it, it's probably broken...

If you're not testing it, it's probably broken...

There's an open-source program, Chaos Gorilla, that will randomly kill an AWS availability zone for you (during normal business hours). This lets you discover whether or not your failover works at a time when it's easier and more convenient to fix it if you discover your failover doesn't work. (Your staff are all in the office, and the availability zone isn't really down so you can always just disable Chaos Gorilla).

It's originally from Netflix:

http://techblog.netflix.com/2011/07/netflix-simian-army.html