Reply to post: On the receiving end of this

‘I crashed a rack full of servers with my butt’

I Am Spartacus

On the receiving end of this

When the Sequent engineer came in to register all of the new SCSI disks we had spent a week installing and configuring. This was in the days when 4G SCSI disks were considered the best thing ever and we had a full rack of these, populated front and back. It ran the Customer Care database, the incident logging DB, and most important, the transaction log that we were required to keep by law.

cue the Sequent engineer kneeling down to read the lower disk IDs, over balancing, reaching out, and yes, he hit the main power contactor. "It was only off for a second", he told me, as the DBA's started wailing that their database had gone down. And then wouldn't come up because it seemed we still had the transaction log on the same disk array.

Cue backup taper recall from the safe store, reinstall, and oh? Where are the historical table spaces? It seems that the Sequent backup program for Oracle doesn't backup readonly table spaces at all.

Next thing happening:

1) Me , explaining to the regulator, why we couldn't fulfil their request for information we had to have;

2) The DBA's doing a repeat after me: "A backup you haven't tested is not a backup at all"

3) Sequent issuing a fix: namely a page to insert in to the manual set saying the readonly tables spaces don't backup.

Very expensive and painful lesson learnt, and we too put cardboard flaps over the very exposed main switches!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon