But never trust the labels....
From experience I have more or less done this and lived to tell the tale...
So I got sent as a junior Unix SA to Milan for a friday/weekend. The jobs was move the 2 Solaris NFS servers from one row to the next. A lovely jaunt and off I go.
I checked and rechecked the labels on the DR server. I even took my packs of WHSmith dot labels in 8 colours to mark the cables and ports on the server. Server at the top of the cab and disks below in a long chain of SCSI chaos. All beautifully labelled.
Shut down the DR server remotely but being old Sun kit it wouldn't power off remotely, so I powered the labelled DR server off. Disassembled it all into a neat pile in the DC.... At which point I rechecked and found that the prod server was dead.
Small panic as I realised that the production NFS sever for a investment bank trading floor was ... on the floor during the trading day. You have never seen an SA move so fast to rebuild a server (at least it was labelled).
Powered it all on. Came up 1st time despite it being red buttoned with live filesystems. Not even a fsck. Went upstairs to check the trading floor. Nothing had even missed a beat. Every trader was out to lunch and their terminals locked. Should have realised that this was Italy, so I did the same.
Post mortem showed that someone had swapped the CPUs of the prod and DR and not the labels, because the prod box wouldn't reboot reliably. Marvellous. Never trust the labels.
Biting the hand that feeds IT
