Black Wednesday: DXC hosting services wonky for almost 8 hours after core switches go rogue The dust is settling at DXC Technology following an outage at its Wynyard colocation site in England yesterday that hosts services for a bunch of FTSE 50 financial services clients, insiders have told us. Primary hosting is done at DXC Wynyard with disaster recovery provided by the DXC Doxford data centre. Before lunch at …
Fun fact: Us experienced ones who have got the hell out of this rotting carcass have noticed something interesting: the hand-off / knowledge transfer is not happening, or if it is happening it's rushed an incomplete.
When I gave notice, I was left alone to finish up what work I could, with a rough customer hand-off dome for one client on the last day.
When a co-worker (a solution architect) gave notice, she was left alone. Then in the last few days it was realized she had 35 current client projects and then all hell broke loose. The hand-offs were rushed and incomplete again.
There is a real "head in the sand" approach going on here... and it's accelerating the clients that are leaving because they are not being looked after!
Bean counters don’t value the revenue attributed to happy customers. From what I’ve seen unhappy customers spend more trying to fix problems their outsourcer insist is not their fault when the customers insourced staff would have easily fixed stuff for vastly less cost as they know the systems and have a vested interest in sorting stuff.
Leaving aside the obvious management foo bar of increasing this quarters profits by sacking your most experienced staff, and, the pervasive incompetence which results from publicly announcing you hate your staff.
Even for a well run site having a 'Backup' anything is asking for trouble.
As soon as something is declared as the backup it gets ignored, testing is skimped, fixes are missed and switch over procedures go stale.
What to do -- run without backups?
No you call one site 'Red', ' South', and the other site 'Blue', 'North' or some similar name of equal semantic value.
The during the time of your lowest load (say second Sunday in the month) you switch from one site to the other -- every month!
There are numerous advantages to doing it this way.
-- Your staff no how to switch over 'cos they do it every month.
-- procedures are kept up to date and regularly streamlined.
-- Small faults and glitches are found as they happen, not the tsunami of bugs which have accumulated since the last switch-over two years ago.
The other advantage is it simplifies maintenance and upgrades.
You apply changes to the dormant site and they take effect on switch over.
If all is well you can then apply the changes to the newly dormant site, if, not
you switch back and try again next month.
And leave it to those fresh faced graduates to fix it.
What do you mean by "real world experience"? Surely everything they use is the latest kit, all from the same manufacturer, all on the same firmware levels and with fantastically written documentation.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2023
