Smoke on the Tyne: Blaze at BT exchange causes major outages across North East England Thousands of customers of BT's fixed-line broadband and telephone services are experiencing major disruption in North East England after an exchange in Newcastle caught fire. PlusNet, TalkTalk and EE customers are also affected, as well as smaller broadband slingers using BT's wholesale network. Specialist ISP Andrew & Arnold …
Modern exchange technology in buildings built more than 30 years ago having anything like a modern fire suppressant system? The fact that they had local hose fondlers on site says probably not...
The calls that I had to field today because of this though....lord almighty...
Only 30 Years ago?
Just for the hell of it, I cycled down to the local exchange building I had always intended to visit*, to find a very ancient sign proclaiming 'Telephone Exchange' above the door, and a date stone in the wall proclaiming it was built in 1966.
Seems about right, as the condition, materials and (weird!) architecture of the building are consistent with the mid-60s. It's so unusual that I may be tempted to make an offer if it ever comes on the market..
Having somewhat nosily viewed the premises, I effed off sharpishly just as the 'No Trespassing' signs suggested I should.
Exchange code obfuscated to protect the guilty, but it starts with 'EA' which should give cognoscenti a clue....
(*Actually, pay homage to as, back on the day, I was heavily into what is now the the 20CN and it's TDM beauty.)
The older ones used halon gases to push all the oxygenated atmosphere out of the building / room and thereby suffocate the fire, along with anyone who remained inside. They were very expensive (halon gases are not cheap, and you need a lot), and bad for the environment (as they damage the ozone layer, I think, techies, please advise). More modern ones reduce the atmosphere in the room to a low level of oxygen, suffocating the fire, but allowing people to survive. They are also very expensive.
I don't know what they have in BT Newcastle, but there are lots of rooms, and any one of them spontaneously combusting could well cause an outage, and shutdown, if only to reduce the possibility of further damage due to electrical fires, metal fires etc.
Reminds me of line from a Victoria Wood sketch:
Posh, Southern Newsreader: "We'd like to apologise to viewers in the North, it must be awful for you."
Halon 1211 (bromochlorodifluoromethane) is/was super damaging to the ozone layer. It was banned in the UK (mainly) in 2003.
Replacement systems exist, there are a few and they are less dangerous to humans in the area where the gas is released too.
I'm sure most of us older techies have some sort of war story about "someone we worked with" that had some sort of run in with a fire extinguishing system in a machine room... or maybe that's just me ;-)
Your comment was fairly accurate last night. BBC news app on my fondleslab had an article about a smoke cloud seen at a shopping centre that got updated to being a fire in a bakery. All dahn sarf. Nothing about this fire though.
Feels like "the heathens can still send packets using their pigeons; we must alert people to a shortage of sourdough".
<troll> I'm surprised they managed to spot an extra cloud in all t'smoke. </troll>
"Your comment was fairly accurate last night. BBC news app on my fondleslab had an article about a smoke cloud seen at a shopping centre that got updated to being a fire in a bakery. All dahn sarf. Nothing about this fire though."
It was on the main Beeb news site when I got home at tea time yesterday. According to my wife, nothing we weren't affected at all. But then we use VM for 'net and landline, both working without hiccup.
i've been involved with a DC build in the last 5 years. Modern DC's normally have a VESDA and use either FM200 (less common these days due to regulations from the EU) or IG55 as the fire suppression. I suppose it depends in what part of the exchange the fire was, even if it had fire suppression it the fire might have been outside of the area covered
I was talking to the machine room supervisor at the place I worked back in the late 70's, and they had one of those Halon systems. When it was first installed, they tested it, presumably with something other than the real Halon gas. The supervisor told me that it is so vigorous that the incoming jets of gas would knock tape reels off of desks. The entire building shook.
We had to be careful when snaking cables under the elevated floor to avoid stirring up the dust which might set off the smoke detectors down there. If two detectors smelled "smoke" at the same time, the whole fire supression system would activate. They had placed colored sticky dots on the ceiling just above where each detector was located so that people armed with floor-suckers would know where to stay away from.
Exchanges host two fundamentally different types of infrastructure:
Transit infrastructure:
where the exchange is a node for transit - this is regional/national infrastructure with mostly diverse and resilient fibre and active equipment. Any one node disappearing off the network for whatever reason will have little impact on the transit infrastructure, as resilient routes will pick up the automatically re-routed traffic.
Local access infrastructure:
where all local access connectivity (analogue exchange lines, ISDN lines, FTTC/FTTH, fibre infrastructure) from the area that it serves converges on the serving exchange (there are some rare exceptions where resilient infrastructure is taken to another exchange - or an alternative infrastructure provider has independent local access capability).
If the exchange is also a parent exchange for some neighbouring FTTC 'child' exchanges, then they would have been affected, too.
This local access infrastructure is what will have been affected in terms of service by the fire.
From experience BT's idea of resilience and diverse routing is not necessarily what you would think.
Had 2 circuits for a network with diverse routing. Initially they routed via different exchanges but both went through the same exchange elsewhere in the area.
Hence when said exchange had a fire (Denton Burn, Newcastle) lost both circuit and access to that network.
Other circuits for other networks were properly diverse and only lost resilience.
BT's response was we only provide resilience to the point of delivery, the circuits may share a common exchange on the route..
Carliol Square exchange originates from the 1930s so may not but up to date for latest fire suppression
When I worked as a security consultant at a well-known company, we did have several different grades of 'diverse routing' available. The top level was one where access to the general comms network was checked to make sure there was no single point of failure at all (very expensive). You gets what you pay for (as long as you read the small print).
Our line that Died is Through KCOM,
Kudos to OpenBreach, this time, they did a pretty good job of re-routing to get the whole lot done in the space of 3 hrs
Still claiming MBORC for a fire in their own building, especially if it turns out not to be Arson, is a stretch
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
