
Please do not call 911 simply to test your phone line
The outage will finish shortly. We are simply testing our systems.
Sincerely,
Your Friendly CCP Senior Electronic Warfare Officer
Residents of the United States woke this morning to widespread outages of AT&T's cellular service. Problems with the telco giant began flooding into crowd-sourced outage monitor Down Detector at roughly 0400 Eastern (0100 Pacific) and haven't slowed since, with more than 73,000 AT&T outage reports submitted at the time of …
> Your Friendly CCP Senior Electronic Warfare Officer
No it was a Singing Tree or a race condition in a Unix based State Estimator /s
Democracy? USofA?
Wikipedia says is better than I can: Fascism (/ˈfæʃɪzəm/ FASH-iz-əm) is a far-right, authoritarian, ultranationalist political ideology and movement,[1][2][3] characterized by a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived good of the nation or race, and strong regimentation of society and the economy.[2][3]
If not for the last part (regimentation ....) I'd say it describes the current USA quite right.
Less so the last couple of years, but during the dark days of around 2016 to 2020, an excellent case could be made. Though, just like a certain German fascist group suffered a couple setbacks along the way, the American counterpart is hoping that if they just keep trying, eventually they'll succeed.
If not for the last part (regimentation ....) I'd say it describes the current USA quite right.
Uhh, no. It does very accurately describe a visible, vocal minority of USAsians (we call them 'MAGAt"s). But they're not in charge. Yet. So stop projecting.
Oh, and as far as regimentation is concerned, I'd say that is quite well underway in our more backwater third of our country.
You only have to look at the quality of teaching, the inability to control gun crime, the number of fairly insane laws etc that are being enacted (or dropped, Row vs Wade) in the US to see that there is a definite spiral of the downwards variety going on. You are legislating/educating/voting yourselves back into the dark ages.
>"If you are an AT&T customer and cannot get through to 911, then please try calling from a landline," the SFFD said
Ah yeah, the copper POTS landline network that AT&T has been systemically dismantling since 2016 because it's expensive to maintain and not a revenue generator. Those landlines. Just use those.
Maybe you can unplug a fax in your office and hook your rotary phone to the last still working line on the block
>But yes, copper POTS is dying in America.
Just another point of ignorance between urban and rural America.
I've got to drive 8 miles to get the first bar of cell service. POTS with DSL is the only land-based Telco/Internet option available. No one is bringing fiber to my location, much less to my residence. $tarlink is an option if I cut down a bunch of trees for a broad view of the sky.
Being a bit of a retro-tech geek... I have two Western Electric 500 rotary phones - one desk set and one wall mount. Pure analog wired service. Those phones have better audio than any of the modern digital wireless phones around the house. Far better than the iPhone / Bluetooth combination when I am in cell range. It is surprising how much we have compromised on audio quality over the years with the so-called modern tech.
You are an example of why we in the US need a federal subsidised push for a national fiber infrastructure. It will be what the national copper POTS infrastructure was in the 20th century, but providing Gigabit (at least) fiber to everyone. Top down, no excuses, it's the right thing to do, and we should start now.
Sheesh. Like *that's* gonna happen in my lifetime!
America has had several "federal subsidised push(es) for a national fiber infrastructure." The corporations took the money and didn't do the installs, then they asked for more money which resulted in not much more fibre.
Then they sponsored laws to prevent communities building their own fibre networks.
The Reacher Gilt approach to communications infrastructure management.
Well, they did build fiber, tons of it remains dark. Interestingly, AT&T being one leader in owning dark fiber.
All backbone stuff, no last mile or well hell, last 20 miles stuff.
They were quite miffed when the US government returned a bunch of leased fiber that they didn't need, leaving AT&T with even more dark fiber.
Until recently, I'd kept my AT&T copper-wire land line, for redundancy and because I have a Western Electric 302 which hasn't required a software update in 7 decades.
Then one day there was no dial tone. A week after submitting a repair request, it was fixed. They'd decommissioned much of copper-wire terminal box down on the main road and consolidated the few remaining land lines in use to a much smaller piece of hardware, and had just missed something when moving my line over. So keeping the land line was expensive, and the redundancy argument weakened significantly by the fact that responding to any copper-wire failures is apparently last on the to-do list at the AT&T service assignment desk.
So now I've found a gadget that will connect to a cell phone via blue tooth, plug into the now-isolated copper wire in the house, and send a dial tone out all the phone jacks, including to that 302. Apart from the above-mentioned AT&T wireless fiasco, the cellular network is now the more reliable thing. So the same cell phone that owns the old land line number and stays at home also serves as redundancy for the wired home internet. When that goes down, the router can switch to tethering to the mobile device's hotspot and keep the home internet connected. Ironically, with AT&T mobile back up, it served that function just last night as the cable company's internet went off for several hours for maintenance.
Verizon sent me a letter a few years ago, telling me that if I didn't allow them to replace my copper landline with fiber (at no cost to me), that I would lose service. I accepted their offer, kept the new fiber landline for a month, then terminated service. When COVID rolled around, I called them back and had them activate the fiber with FIOS internet on it.
My landline is now through a VOIP provider, over that fiber, can use both pulse (rotary) and DTMF phones on it, and the number is the same one I've had for 40 years, at a small fraction of what a Verizon landline would cost me. It works just fine, and I used it to verify that I could call my AT&T cellphone this morning. Whatever rot is in the AT&T infrastructure, it hasn't yet made it to my neck of the woods
"Until you have a powercut and discover that the battery they included in the modem died years ago. or you find that the cabinet at the end of the road that the fibre terminates at doesn't have battery backup."
And that the included battery, if working, still only gives a couple of hours of service before discharging. Had to put a massively oversized UPS onto mine to retain service during an outage during hurricane Sandy.
Coincidentally, I was listening to NPR this morning, and they had a segment about AT&T's efforts to abandon copper. There was a statement from AT&T along the lines of "oh just switch to cellular, that works just as well".
I glared at my radio and muttered "how many E911 outages have happened on your cell service in the last few years.". Then I hit the preset for a music station. Their morning show jock was warning people that AT&T was working in our area, except for 911 calling.
AT&T used to have a network literally built to survive WWIII*. Shame how far they've fallen.
*look into AT&T long lines, lots of Reg readers will enjoy the history.
One of the issues here is that the current AT&T is not really related to the AT&T from decades ago outside the brand. I have trouble believing that people who remember those past decades associate AT&T with good things (it was broken up for a reason), but apparently the branding people did.
Cellphones aren't tied to their primary network for calling 911; this is built into the cellular standard. In fact, you can call 911 without a SIM, as long as your cellphone can see a network. So if people are unable to call 911 but there is still an active network (of any description) in their area, it'll be one of two things:
1. the problem is indeed broader than 'just' AT&T, and other networks are experiencing problems too.
2. OR the cellphone thinks it's still connected to AT&T and therefore won't try other networks automatically.
If 2., then removing the SIM should allow you to call 911, as the cellphone will then not remain tethered to AT&T, but will default to whichever (working) network it can pick up.
This is all true, but I think it's easier for them to just say "have a neighbor call it in" or "time travel to 2009" than get into SIM support calls on the 911 lines. Can you imagine the number of dowagers sitting at home with their cell phones in ruins alternating between 911 calls and bothering their grandchildren?
"I do have the simp right here. It says Sandisk."
"Alright I've rebooted the phone, what do I do next? Hello? Hello?"
"I just wanted to call back say thank you for the patience you showed when you spent 90 minutes fixing my Tracfone. 911 operators are the best."
I think it's likely a combination between someone being in an area where other providers' signal is weak and having phones that lack some of the bands that other providers would be using in the area. If someone on a different provider would be using a band that wasn't included on the device because that one was built for AT&T networks, they would be unable to use that one to connect to emergency services. This would indicate why it was just some customers, not everybody in the city, who can't connect. It would be easier to demonstrate if we checked the devices that didn't work. Most flagship devices have a lot of bands, but some cheap ones, especially the cheap ones sold under the providers' brands, go for the basic minimum.
In (at least) the Netherlands, you absolutely can.
Call 112 without calling credit and without a SIM card
You can always reach the emergency number 112. Even if you call with your mobile without calling credit and without a (valid) SIM card. All 112 calls are given priority over other calls. Did the connection suddenly drop? The central control room employee will then call you back if your details are sent with the call. Your details will not be sent if there is no SIM card in your mobile phone.
Not sure I understand the penultimate paragraph completely.
The towers are just metal. Yes, they're owned by tower companies, but the towers themselves (as long as they're standing), and who owns them, have no effect on the cellular service provided through the equipment bolted to them and in the shelters at their bases. THAT is owned by the cellular providers, who pay rent to the tower companies for space on the towers.
I doubt very much if the ownership of the towers has anything to do with this outage. A network misconfiguration would be my guess.
The towers are just metal. Yes, they're owned by tower companies, but the towers themselves (as long as they're standing), and who owns them, have no effect on the cellular service provided through the equipment bolted to them and in the shelters at their bases. THAT is owned by the cellular providers, who pay rent to the tower companies for space on the towers.
That's how it worked yonks ago when I designed cellsite radio gear. I'm VERY skeptical that it has since changed so that now companies like American Tower own and run the actual radios and telephone networks — and that Verizon, ATT, and T-Mobile are just reselling access/minutes/GB the way that Cricket et alia do.
Cellular network operators have been using rented antenna and equipment space on/in other people's towers and buildings since the very beginning back in 1980.
Cellular network operators have been using rented antenna and equipment space on/in other people's towers and buildings since the very beginning back in 1980.
True for MVNOs, but the tower next to me has three separate sets of antennas, all (as far as I can tell) owned by different companies, and upgraded/changed at different times by different crews. I think the current users of "my" (actually my neighbor's) tower are AT&T, Verizon and T-Mobile
Tower companies just want an income stream, they don't necessarily care what's on the towers. The problem here is that they also don't want to do preventive maintenance or be too picky about loading stresses. I'm not sure the tower next to me has had any engineering work done to verify that its current loading is within limits...but I'm outside the drop radius, so it's just casual interest on my part, not panic.
According to BOFH excuse generator it was a Solar Extrathermal Excursion
In my neck of the woods the local cable company would take an unnegotiated shortcut.
Twice a year the satellite you are getting your shows from passes in front of the sun and your antenna sees nothing but sun.
Nowadays most satellite companys put up a 2nd antenna and point you where the sun is not.
Back in the day, cable companys were supposed to record at a different time of day and play back from the good source.
When the company couldn't be arsed you would see the show fuzz out and disappear. If you called the cable company, they would call it sun spots. Close I guess? The antenna was spotting the sun.
Icon because that is what the antenna sees.
I'm in Eastern North Carolina have ATT cellular.
My phone has been working fine, no problems, and I've heard plenty of reports of people with ATT phones who also haven't been having problems. However, my car had no connectivity, and I believe it connects using ATT's LTE network.
From what I've been hearing is the problem is with the LTE network, my phone is on the 5G network and has been unaffected.
Thanks El Reg for the quote "the application and execution of an incorrect process as we were expanding our network."
Yes, when you look at the map of the problems it's pretty clear that "A bad software or system update" is probably the cause so it should be much easier and faster to fix than any malware issues - hopefully the problem is ending now.
The morning usually starts with junk calls -- real people use text or emails.
Today has been quite peaceful. Maybe its just wishful thinking on my part because I'm a T-Mobile customer. Maybe not, maybe the spammers all prefer the AT&T network.
Back in the Good Old Days the Bell System was very prosaic but primarily designed for reliability. I'm not so sure about modern cellular equipment. I've always treated cellphones as 'nice to have' but since coverage where I live has always been hit and miss (mostly 'miss') its more useful when we go to the urban areas and experience real coverage. (We usually got our cell service through WiFi at home but once our Internet provider started reselling cell service our phones became unreliable. A picocell fixed this.....but.....we shouldn't have to have to mess around like this for an essential service.)
We had a multiple-day power outage in July a few years back when we were living at the Stately Manor. There was several hundred dollars' worth of meat in the big freezer that we really didn't want to lose, including some that wasn't trivial to replace (fish we'd bought Up North, prime rib from the county fair blue-ribbon cow). But the local stores were out of dry ice by then.
No one had cell service — tower batteries were dead. But we still had a landline with a "string-a-ling phone", as Wonderella once put it. Entirely powered by the line current. So I called around and found that one of the local convenience-store chains would sell dry ice out of their warehouse, which of course had its own generators for outages. They didn't advertise it, but they'd do it, provided you wanted to buy at least $20 worth and would pay cash.
That one trip alone made that phone line worth having.
(Here at the Mountain Fastness, getting a landline run would be wildly expensive, and I expect it wouldn't be much more reliable than cellular is.)
Indeed. My copper based landline that worked perfectly for decades is now a copper based "fibre" connection over the same decades old copper wires and now stops working every time it rains or the power goes out, thanks to BT Openreach and Zen Internet.
There's no timeline yet for anyone to provide real fibre here, if they even know the difference, as we're only a small city. :-(
Sure, outages in major cities, but what calamity has wiped New England, most of New York and Pennsylvania, Michigan, and Indiana from the face of the Earth? That seems like an important aspect of the story that no one's reporting.
(I mean, if it was just, say, New Hampshire, we could deal. But that's a lot to lose at once.)
I have been a victim of AT&T for years going on decades. I had resisted the fad of first holding a brick to the side of the face from mouth to ear, and later doing the same with a Star Trek communicator-esque flip phone. My employer back then had decided all support roles were required to be reachable anywhere including me which worked within a space where cellphones were verboten. They issued me a Motorola flip phone with service by Verizon. Then after the numerous "me too" company issued cellphone requests became burdensome, they changed to reimbursement. After having the Verizon phone for what I considered a long trial period, I had found that stomping grounds were poorly served by Verizon but well served by AT&T this included my mother's house where Verizon phones killed their batteries trying to find a tower. I visit mom often enough that it matters. So when I had to contract for my own cellphone service at the company's expense I chose AT&T in order to be somewhat useful.
Even at home AT&T had better signal than Verizon, more "bars" in general and I could go to the basement without losing a call. It was almost better than tolerable back then. When AT&T began building it's 5G system, signal at my place began to degrade. At times worse than Verizon had been. At mom's Verizon was still non-existent but AT&T was degrading as well, sometimes no signal, sometimes poor call quality, sometimes dropped calls. To this day it has only marginally improved. BTW I am determined to stick with my "grandfatherd" "loyalty" plan which is only 4G but avoids the 5G tax. As I hear from others 5G in my area is hardly an improvement even non-existent in some spots. I have no choice, I could go with a MVNO but that does not change the service coverage, but maybe time to check into the option again for other reasons.
AT&T does provide excellent customer dis-service though. Calls to complain connect quickly and are auto answered promptly with the finest examples of maladjusted proprietary hold "music" audio. These calls do not drop either while on hold. Many times the calls will even stay connected through multiple transfers to different departments where audio quality is remarkably diminishing at each transfer. Other times a transfer goes to an anechonic chamber where the silence is deafening or possibly the fates will decide the call should just drop. All the while the voices on the other end are frequently accented perfectly mismatched to one which will not be intelligible to the caller.
The fixes presented during these calls involve, restarting the phone, shut down removing and cleaning the SIM reinstalling and powering up, refresh the account provisioning and pushing related config to the phone, issuing a new SIM, use WiFi calling, try on another phone, unplug it and plug it back in, try holding it differently, fully drain and recharge the battery, fully charge the battery and redrain the battery, remove the protective cover, remove the screen protector, remove both the protective cover and screen protector, try from outside, try from the attic, try from atop the chimney, cut down the neighbor's tree, cut down the neighbor's house, cut down the neighbor's SUV, does the neighbor have a windmill, does the neighbor have solar panels, does the neighbor have a daughter, power off and then power on the house, sunspots, moonspots, leprechauns, ok try that and give it a day or so and call back for more help if that does not work. Please take the survey at the end of the call.
The online experience is equally wonderful. Second Factor Authentication or is that Multi-Factor Authentication, has become mandatory. I remember when username and pet's name were all that were required. Shortly after some high profile SIM jacking events they did provide an optional security code. And eventually the pet's name had to be longer than Dino then longer than Flipper then longer than Tralphaz so currently Mephistopheles looks very confused when I call them to the dinner bowl. Maybe I should use a more secure passphrase like Satan's Little Helper. The 2FA is very sophisticated and therefore secure, so I have been told in an accented voice. You an only receive the randomly generated code via SMS, but wait not just any SMS, only to a line number on the account's plan. My concerns of being unable to receive the code because of: 1) being out of coverage area yet have internet access; 2) having only one line on the plan but having a malfunction, pilfered, lost, destroyed device; 3) having internet access yet the cell service is down; or 4) other reasons I cannot disclose; have been politely and cheerfully dismissed.
Real fun ensued when attempting to login online when traveling and therefore using a VPN for a secure connection. The main website would appear just fine. The convoluted authentication workflow involved redirects to servers protected from hackers by denying connections from known VPNs. I found out through my own troubleshooting changing the pet's name yet again while on travel without the pet was not feasible. Many other websites had also begun using the very same hacker protection, so I now use an undetectable self-hosted VPN.
For the privilege of having to endure these fine levels of dis-service excellence one must be prepared to pay the piper, never mind the piper, no longer enough left after paying AT&T with rate increases at or above the rate of inflation.
On a related topic: New landline service is no longer available in this area. I only consider PSTN service to be landline. And even for those lucky enough to still have it or acquire it under extreme circumstances, the cost has risen to match or exceed cell service and reliability has degraded to cell service level or below. VOIP is not landline. Cable fixed line comes the closest, but requires power locally.