Major telco outage leaves millions of Australians disconnected Australia's second-largest telco, Singapore-owned Optus, experienced an outage beginning Wednesday around 4am Sydney and Melbourne time that left millions nationwide without phone or internet access – either mobile or terrestrial. The outage also affected transport as Melbourne trains were temporarily inoperable and network …
You beat me to it ...
Though, at the time it certainly wasn't funny. I was out in the garden this morning when two police cars and 4 coppers arrived, because the social worker couldn't contact my elderly neighbour on the phone (and the 4 large police couldn't get a response from knocking). They got through in the end , and she was OK - but I'm sure similar stories have been happening all over Aus today.
What staggers me is the sheer amount of critical public infrastructure that apparently doesn't have a fallback mode and simply fails when it loses comms.
Remember, the Internet routes around problems. Except when the routers are sulking. I suspect the problem will end up being a BFR for everything design, ie over relying on routers for both IP and MPLS services. That can mean an over dependence on the router for routing, and 'transmission' services like keeping pseudo-Ethernet services running. So a bad day on the router, and everything collapses. Much safer (and often cheaper) to let routers route, and switches switch. Especially when switches can usually switch far faster than an IGP or EGP like BGP can notice.
Witness the danger of convergence. The 'digital revolution' has replaced several entirely independent comms systems, with its associated resilience, with everything depending on a single infrastructure, so everything dies when this breaks. That's not really progress from the human perspective -- indeed it presents a real danger (even to life and limb).
If I had read this 2 months ago, I probably would have made some snarky comment about, Millenials not carrying cash, and how I always keep cash on me in case you come to a place that doesnt accept cards or an emergency like this happens.
BUT I actually just got back from a month in Aus, and wow, I have never been to a place where cash is used LESS than in Australia now! I always had cash on me, but a LOT of places wouldnt even take cash, even for small purchases. Everything was digital and over Eftpos. Probably because finding an ATM was also a struggle in a lot of places (or with massive fees piled on top!). Even my 80 year old mother, didnt carry any cash anymore.
So when the network goes down like this, a lot of the small shops would be screwed! First, they cant process payments, and second, their customers dont have cash on them to make the payments in another way. This will have hurt a lot of small businesses. Will they get any sort of compensation? Not f%&kin likely...
Almost like if you're running a business-critical system like payment processing that you need an independent backup that's not reliant on a single telco.
No matter the practicalities of that, small business is no different - if it's going to cost you, say, thousands of dollars to go a day without payment processing, then you should at least be spending a thousand dollars to try to mitigate that in most circumstances.
They could buy Starlink (or other satellite broadband), they could use another cellular provider, they could link with other local shops and get another / better line, etc.
I cannot imagine - as a self-employed person - running any kind of physical retail shop without at least a Square or iZettle reader under the counter for backup, and some other way of getting online.
I kind of get it in the middle of nowhere out in the sticks, but that's not where the vast, vast majority of Australians live or Australian shops are.
Honestly, if I had my entire income stream reliant on a little box that swipes cards, I'd have several backups of all the parts necessary if that were to fail.
> a LOT of places wouldn't even take cash
Well then as I see it, it's their own damn fault, and I shed not a tear for them. Taking more than one form of payment is just plain business sense.
It's like all the websites that used to just take Paypal only. Nope, I don't care how unique and great your product is, I'm not touching Paypal with someone else's 3-metre pole.
We've had a couple network outages here, and some of the restaurants even hauled out the old slide receipt embossing machines... the ones where you put the card over carbon paper, slide the wiper and it embosses the card number into the receipt.
When it comes down to your money and your livelihood, you need to be resilient and not a stubborn asshole.
Cash is getting expensive to accept because the banks are closing all their branches.
Small businesses have nowhere to deposit the day's takings or to get cash float for next week.
A big supermarket can afford to pay for a daily visit by an armoured vehicle, but a small business cannot.
So accepting cash often means having to keep a huge amount of cash on-hand simply to operate - making them a target for burglary and theft, generally uninsureable even before considering the risk to staff safety.
That's why they don't take it.
Cash is getting expensive to accept because the banks are closing all their branches.
Then why the hell do they rip you off with card fees?????
And the ones that really piss me off are those that want me to go online to order from a menu and then enter all my details or worst download an app, No way.
As an IT manager for schools - that's a nonsense.
There's no way that an Internet failure should take out fire alarms - they can't be compliant.
Also - no MIS access is feasible, but emergency contact details should be printed out and/or otherwise held offline and on-site. You could phone the MIS provider and ask them to send the output of a single emergency report (which all MIS have) to an email address, and then access it on a phone, in extremis.
Stopping a school running for either of the above is a failure of their basic processes, whether they use third-parties or not.
What are they going to do in a real fire that occurs in the middle of the school day and burns through the networking cabinets? Are they going to shrug and say "Oh, well, we can't possibly know how to contact parents or see if we're missing any children now". They'd be shut down by the DoE and the fire service if nobody else.
And no - repeat NO - fire alarm should have any network dependency, let alone Internet dependency. You keep that stuff entirely separate for a reason.
Something tells me a fire inspector is going to be paying them a visit "real soon now".
Given how well the negotiations went to allow hassle-free roaming in remote areas a couple of years back I can't see them making more headway to revisit that with the added inclusion of metro.
What does need to happen is for whichever regulator is appropriate to be given the teeth to go and make it happen
Beggars belief that emergency calling - which has been with us now how long - 100 years? had no backup in place (few posts above).
Which traditional system of communications worked? Genghiz Khan had Pony Express type messengers to transit the whole of Eurasia. The English system of hilltop beacons worked pretty well. Otherwise we’re back to jungle drums.
Welcome to the future and the bigger problem is after the expletives we shrug and go “oh well”.
I do.
Tech is fragile. Much less resilient than physical alternatives. Yet nobody seems to have any form of Plan B.
In the UK a school had to shut because of a 'system failure'. It's pathetic.
https://www.bbc.co.uk/news/uk-england-cambridgeshire-67314252
BT are getting rid of UK landlines that work in power cuts, just as climate change is causing more of them, just so they can save cash, their chums in government supporting them.
Every service and institution we have seems to be run by greedy, incompetent morons.
They all probably had Optus mobile and Optus NBN at home.
I'm tipping they use a cloud hosted collaboration and/or email service.
I wonder if they still carry old skool pagers - even if they do I'd also bet they rely on an internet gateway to send pages.
Can you imagine the reaction of the techs after they rolled out yet another screwed up manual change or buggy firmware.
"Oh crap, I better call my boss...... d'oh!"
The first 3 hours were probably waiting for people to come in to the office....
I think people are locked on the BGP because that's what caused the Facebook outage.
Another possibility is a private root authority certificate expiring. Suddenly all parts of the network that rely on PKI would instantly fail and this would cascade into automatic provisioning failure.
Normally it wouldn't have been too much of an issue for me, as internet is with another provider, but a couple of hours into the outage internet went down too - a local power outage that was fixed 9 hours later. So no mobile phone; no VOIP as internet's down. Coffee shop (run by a mad French ex-gendarme) was taking cash as usual and had their bank account details shown for direct credits so that was OK.
Reminds me of when I worked for an ambulance service, and we stipulated that all infrastructure must be in-house. It all crashed when the Health core router died even though none of our traffic was supposed to go through it, so paper and pen for emergency 000 calls.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
