Turn it off and back on
You know, I've seen this actually work so many times I've stopped making fun if it.
Oh, and "told you so".
If Vulture South wasn't running short-staffed yesterday, we'd have stayed with the Senate Committee hearings inquiring into Australia's Census outage on August 9, and caught this gem: “If we had our time again we would have tested a hard power it off, power it on that router, that would have discovered earlier that we had that …
The main reason it works is because it resets everything into a known state and any issues related memory/resource leaks, conflicts, stale configurations or in memory data corruption are cleared away (leaks will generally require periodic reboots - I have resorted to scheduling reboots/restarts when clearing memory leak isn't easily solved).
Often it would be possible to avoid the reboot (assuming you have sufficient permissions) - but it is much easier and quicker to tell most users to just restart the computer and doing so it will have cured most user immediate problems without getting "technical"
Back in the day...
Had a data-logger running on a Win95 PC (yes, that long a go) - but because Win95 had memory leaks and because they client never bothered to switch off the PC I had to set it to auto-reboot at midnight to reset everything.
So not quite "switch it off/on to reset", but close enough...
If they used IBM, probably.
Honestly "Big Blue" wonders why people increasingly avoid it, simple - when they are trusted they royally balls it up.
This census was a big deal, they cut corners to save a few quid. Aussie government should ban IBM from procurement for a 10 year period.
“If we had our time again we would have tested a hard power it off, power it on that router, that would have discovered earlier that we had that reboot and configuration loading problem”
I know roughly what this means, but is this what passes for a coherent sentence these days?
"IBM and the ABS managed to put in place another howler of a misconfiguration: “Island Australia” (which only partially worked) was designed to block traffic from outside Australia, but IBM was hosting the Census password reset facility offshore."
OMG - I laughed so hard my wife was like "WTF?".
Sometimes the weirdest shit is more stressful than "yank the power"
I have a Linksys WRT1200ACv2. It has no problem with "yank the power"
Pull out the WAN cable, however, and it loses its shit. It changes its IP to a random 192.168.x.x or 10.0.x.x subnet, loses all the DHCP reservations, and loses all the opened ports in the firewall. The only way to find it on the net is to do a DCHP request and find who answers.
Very annoying and I only discovered this long past the time I could return it to Best Buy. Installing openWRT fixed it though.
The majority of actual IT function for IBM Australia is provided by IBM India. It was likely a calculated risk, locking out non-Australians would knowingly impact their offshore operations, but having no resources locally to perform admin work anyway, it was something they just had to accept.
Unfortunately I've been involved in more than one "The running config wasn't saved to the permanent config" while working at IBM, it's something all network operations people should be entirely aware of by now, and is common enough and avoidable enough to be entirely inexcusable.
Biting the hand that feeds IT © 1998–2020