Power, internet access knackered in London after exploding kit burps fire into capital's streets

Some of the UK's ISPs may want to rethink their routing schemes after a massive fire near Holborn tube station in London knocked out power and internet access across several regions of the capital. There are now around 70 firefighters at the #Holbornfire no reports of injuries and local offices have been evacuated pic.twitter …

[JDX]

One of those newfangled telephones?

[Anonymous Coward]

How can people log in to El Reg forums if their internet is down, eh?

Well they wouldn't know the story is happening to comment if they did not have another way to go online.

Ever hear of browsing using your phone? wifi?

good god these people don't ever think before commenting, and they are supposed to be techies?

hope you never work in my building

[Anonymous Coward]

Re: How can people log in to El Reg forums if their internet is down, eh?

Or they were just joking.

Pretty sure nobody wants to work in your building if the requirement is to have a stick up ones bottom.

[Destroy All Monsters]

There goes the cloud, in the shape of belching smoke

Ever hear of browsing using your phone? wifi?

I hate to break it to you, but these systems actually use cabling to bring the data to whatever exchange point it needs to go to.

hope you never work in my building

Is that the one where unicorns magically radiate data to the destinations, powered by bionic carrots up their arses?

Go to bed, PHB!

[billse10]

Re: There goes the cloud, in the shape of belching smoke

probably didn't hurt as much as those carrots.

[Tom 13]

Re: There goes the cloud, in the shape of belching smoke

Upvoted for the title alone.

[Suburban Inmate]

Re: There goes the cloud, in the shape of belching smoke

Is that the one where unicorns magically radiate data to the destinations, powered by bionic carrots up their arses?

There's an oatmeal comic right there!

[The Vociferous Time Waster]

Re: How can people log in to El Reg forums if their internet is down, eh?

"Ever heard of browsing using your phone? Wifi?"

Ah, the pseudo techie.

I suppose you think those technologies work through magic and done have wired backhaul over the very same wired infrastrucure.

Go back to your html 'coding', mate.

[Tom 13]

Re: work through magic

But, but, but, but

[loud waily type voice] IBM said HAVE pixie dust in their commercials! [/end waily type voice]

<<sob>> <<sob>> <<sob>>

[Nathan 13]

Home ADSL OK in NW6

And Three mobile data is running at 0.2mbps instead of 0.1, must be using the spare bandwidth lol

[Anonymous Coward]

Re: Home ADSL OK in NW6

Net access fine in south Devon too. Must be a pretty localised issue.

[Bilby]

Re: Home ADSL OK in NW6

No problems here in Brisbane, Queensland, Australia either, so it does not appear to be affecting the entire planet.

[Allan George Dyer]

Re: Home ADSL OK in NW6

Fine in HK too, so yeah, pretty localised.

[Anonymous Coward]

O2

O2 have lost a bunch of cells in the area.

[hplasm]

Re: O2

Brain cells?

[John Brown (no body)]

Re: O2

"Brain cells?"

Don't be silly. They never had that many to start with!

[a cynic writes...]

We just went home...

We lost the fibre (and SIP trunks) to begin with. The power went sometime later.

We've already been warned that the fibre is likely to be down over 24 hours, so if we do have power in the morning then I get to finish off reconfiguring the firewall to use the old ADSL set up. If not then it's VPN direct to the data centre for our key people and I can kiss goodbye to taking next week off.

Joy...

[Anonymous Coward]

Life is just not fair is it!

No problem here in rural, isolated Cornwall. We always miss out on the interesting stuff

[Matt 21]

Re: Life is just not fair is it!

Last time I was in Cornwall the only way I could get a mobile signal (in the area where we were staying) was to stand a yard or two in the sea at low tide with my arm in the air.... which suited me perfectly for a holiday.

[Anonymous Coward]

Power circuits are currently be reconnected to bloody huge generators. Looks like this could be a long outage.

From UKPN:

Update at 19:13 - Our engineers are continuing to work with emergency services to tackle a fire in a tunnel in the Kingsway area. At this stage our plan will be to restore the majority of customers by using several large generators. However due to the severity of the fire and the developing situation it is unlikely that power will be restored tonight. We apologise for the impact this will have, we'll work to restore power as quickly as is safely possible.

[x 7]

Oh shit........presumably its cables in the WWII Chancery lane Citadel - which for a while held the Kingsway phone exchange. There are miles of tunnels down there. Putting it out is going to be really difficult - theres (apparently) now only one access and simply just reaching the underground tunnels is going to be hard.

Theres a risk the fire could leak through into the tube as well if it can break through the sealed passageways.

Potentially this is a nightmare for the fire crews which could run for days

[wolfetone]

I'm not sure if this is true or if this is a very late April Fool's joke...

[x 7]

details of the citadel are at

http://www.subbrit.org.uk/rsg/sites/k/kingsway/

[Anonymous Coward]

The former Kingsway exchange isn't near the road of the same name, it was apparently named deliberately to sow some seeds of confusion during wartime. The tunnel network doesn't carry power cables.

There is however a tunnel under Kingsway that does carry power cables - the old Kingsway tramway tunnel. Lots of work going on in there for Crossrail too.

[x 7]

from what I've seen of the various films / photos the fire looks too far north to be in the tram tunnel?

There once was a plan for a Crossrail route which used both the tram tunnel and the Chancellory Lane shelter tubes - it would have been an interesting bit of tunneling to join the two. 90% bend and 100 feet difference in depth to negotiate in around half a mile, constrained by the buildings along the route

[Anonymous Coward]

The tram tunnel extends up to Southampton Row and then runs all the way under Kingsway.

[MattW99]

No problem

Everything fine up here in the grim north.

[cosymart]

No Problem

Everything fine in the grey area between the grim north and the effluent south.

[h4rm0ny]

Re: No Problem

Not sure if deliberate or accidental, but I gave you a thumbsup for "effluent South".

[Anonymous Coward]

Re: No Problem

shurely you mean "Affluent" south - Ed (:

Pmsl

[Peshman]

Re: No Problem

I think you've just pointed out that AC is in actual fact Alex Salmond.

[Anonymous Coward]

Just seen this on BBC news ... interview with distraught women wondering how she was expected to cope with the situation as she had no internet!

[billse10]

newsreader interviewing their own producer, by any chance?

[Mark 85]

Probably had a truck load of cat pics to upload to Farcbook.

[wolfetone]

Probably interviewed the cleaner again:

https://www.youtube.com/watch?v=zWAvHnfJsOQ

[x 7]

checking online maps, the fire was apparently first reported at the "Upper Tribunal Immigration and Asylum Chamber" in 15-25 Breams Buildings - which backs onto Took Street, site of one of the citadel access shafts (now capped over). Fire fighting in closed tunnels - not good. It took days to put out the similar underground Guardian exchange / citadel fire in Manchester ten years ago

[Terry Barnes]

Pictures on the BBC website show emergency services folk congregating by the entrance to the Kingsway tram subway. Isn't it more likely that a fire is in there?

[Tom 13]

Re: It took days to put out the similar underground

On the bright side, that's much better than the Centralia, PA fire. That one started in 1962 and they think it might last another 250 years.

But seriously, I hope everyone is okay. I know these are the kinds of fires where despite the best training and efforts some of the front line guys always seem to get injured.

[razorfishsl]

Maybe some fire-doors and suppression systems would have been in order

[Anonymous Coward]

The problem there is that having the systems is not enough. You have to have people that know what they are doing to maintain and test those fire doors and suppression systems. BTW, it's handy having them available when a "situation" arises.

[Mark 85]

Fire door in a tunnel works well... except the cutouts for the cable to pass through.

[Me19713]

I work for a power utility in the US (37 years and counting). We ran a lot of fiber in unused power and gas lines in urban areas. Early one morning the substation operators were busy re-energizing some manholes after a fiber-pulling session.

We had a phase-to-phase fault (34 KV, IIRC). A 36" manhole cover was blown into the 4th floor windows of one of the local banks. No one was hurt, but the fireball and falling debris tore up a few cars. And there were more than a few broken windows (and dirty shorts).

That certainly broke the monotony.

[Anonymous Coward]

That certainly broke the monotony.

That's an almost British understatement :)

[Jellied Eel]

That's one of the things that put me off working in telcomms for a power company. And seeing a report from a field engineer cancelling a request for a TDR team to shoot a fibre fault. He didn't need it, he could see the cable ends in the crater. Oil-filled HV cable had a bad day and made quite a bang.

[Winkypop]

I told you, I told you all

The Internet is the devil's work!!!!

See; flames from below!!!

[WookieBill]

We are on high holborn :- currently looking like we will be down another day

My companies head office is on High Holborn. We lost power at around 12pm yesterday. Once our emergency lighting failed after 3 hours we evacuated (to the pub over the road which still had power). The on-site UPS had a fairly short battery life so our domain controller on-site was not able to be shut down cleanly and we are a little concerned about some of the switches coming back to life or not. Our company is citrix based and most stuff is tucked away in a data center, so loss of systems is not a major worry. Unfortunately, our CAG was not provisioned for DR, as a result we only have 40 remote access licences and far more staff than that, so for most people, working from home is not an option. Also, from the last update from building management, we are unlikely to have power tomorrow either, as they can't get into the tunnels under Kingsway and are going to have to dig up the road. It's going to be interesting!

[Anonymous Coward]

Re: We are on high holborn :- currently looking like we will be down another day

Unfortunately, our CAG was not provisioned for DR, as a result we only have 40 remote access licences and far more staff than that, so for most people, working from home is not an option

Let me guess: none of your upper management has ever heard of ISO 25999? That's going to get both interesting and costly.

[richardcox13]

Re: We are on high holborn :- currently looking like we will be down another day

>f ISO 25999?

What's that?

Or are you mixing (now withdrawn) BS 25999 and (its replacement) ISO 22313?

[Anonymous Coward]

Re: We are on high holborn :- currently looking like we will be down another day

Absolutely. Serves me right trying to use the Innertubes whilst high on a heady mix of flu medication and strong coffee.

[kaiserb_uk]

Verizon MPLS

Verizon MPLS connections affected, we've lost two sites in Luton and Portsmouth...

[x 7]

Assuming the fire IS in the Ciitadel, and assuming that access hasn't changed since the SubBrit report, then getting into the tunnels is going to be interesting. The tunnels are 100 feet deep. The only open access was a lift shaft at Furnival St. No-one is going to want to use that. Stair access at Chancery Lane station was sealed off a long time ago, while the stair access at 31-35 High Holborn (the original Chancery Lane station building) was blocked more recently.

I can't see anyone allowing the tunnels in the current station reopened. So the only foot access is likely to be through 31-35 High Holborn IF the stairwells can be reopened. It may be preferable to drill a new shaft instead.

Whatever happens, the tunnels will need inspection to see if any damage threatens the buildings above.

And an afterthought - when the "sister" site in Manchester - the Guardian citadel - went up in flames ten years ago the authorities were very loathe to admit the facts. Expect the same here -there is still a strong aura of secrecy about these citadel sites, even though they are redundant. At Manchester the press reports spoke of a "cable duct" catching fire, when in fact the whole site went up. I recognise the same kind of reporting here, with the BBC reporting a "cable duct" on fire.

[hplasm]

"...there is still a strong aura of secrecy about these citadel sites,"

Battle beneath The Earth...

tell no-one.

[Anonymous Coward]

"Assuming the fire IS in the Ciitadel"

That's a pretty big assumption. Electricity companies have their own tunnels for HV distribution around London - pylons being a little impractical. Why would anyone go out of their way to route power cables through a disused underground telephone exchange that doesn't actually go anywhere?

[x 7]

"That's a pretty big assumption"

Fair comment. But looking at the various films and photos its clear something more than just an HV line is on fire. Those flames coming out of the ducts are are gas flames, presumably from a fractured gas main. And how do you account for the loss of telephones and the wider-scale internet outage? And the official statements this morning that it may take "several days" to resolve the fire? It may well be the citadel wasn't the seat of origin - but my guess is that the fire has spread to it. However, I hope I'm wrong.

[Anonymous Coward]

"Fair comment. But looking at the various films and photos its clear something more than just an HV line is on fire"

The fire brigade have said that the fire is the result of a gas leak which has been ignited. They can't put the fire out until the gas is turned off because that risks an explosion if unignited gas is allowed to build up.

http://www.london-fire.gov.uk/LatestIncidentsContainer_01030482.asp#.VR0ZeOFIlfA

The loss of Internet will be down to the cutting off of power - not all telcos back up everything with generators - and in the case of companies like Verizon and Colt it'll be because their privately owned fibres are routed through the same tunnels - there's a lot of telco cabling running through the Kingsway tram subway.

Citadel / Kingsway was built to be impervious to external events - bombs, fires, military action - it seems pretty unlikely that an unrelated fire nearby is somehow going to leap through earth and mud and then through a cast iron tunnel wall into a different set of tunnels.

If Citadel was on fire, Chancery Lane tube station would be closed. It's not, trains are running normally.

[Jellied Eel]

Problem is back in the day, there was pretty much just the BT ducts and other utilities. Now the area is a major cable jungle with multiple ducts with mutiple paths for gas/fire to travel. There are power tunnels, sewer tunnels, ducts connecting buildings. There are cable chambers connected to other provider's duct networks. Those *should* have gas stops, but if they don't then gas or fire can travel around easily.

So there'll be a lot of field engineers looking for fibre routes around the location to bypass damaged sections and restore services, then a lot of work to replace any damaged sections once the area's declared safe.

[Will Godfrey]

Famous Last Words

We need to cut maintenance costs, so we might as well seal that entrance - We're never going to need it.

[Anonymous Coward]

Re: Famous Last Words

"We need to cut maintenance costs, so we might as well seal that entrance - We're never going to need it."

Not before I get my megabonus for costcutting anyway. And when the sh1t does hit the fan I'll be long gone, wreaking similar havoc on some other unlucky outfit who won't have seen my track record.

[Anonymous Coward]

Fire still going

Both of our office networks were taken out yesterday (chancery lane) by this so most of the company is working over VPN today. Office still had power last time we checked.

One of the fires was put out pronto but apparently a second one is still ongoing and they suspect it may be being fuelled by a broken gas main. As such carbon monoxide levels in the tunnels are at crazy lethal levels and BT are apparently unable to re-route as a result.

So given the extensive fire damage I suspect networks in the area will be down for considerably longer than 24hrs and TBH I'd be surprised if there was any before monday...

[kidtrebor]

Short power outage here on Pall Mall over-night, looks like it kicked in around 9pm.

[Winston1984]

It's just a small utility subway

The tunnel involved in this hiatus appears to be neither that of the old Kingsway telephone exchange, which is located underneath Chancery Lane tube station, nor the Kingsway tramway tunnel, which runs under the middle of the road. It looks as though it's one of the pipe subways built by the London County Council in a few streets between 1861 and 1933 to carry gas and water pipes. Since then other utilities' pipes and cables have been added. AIUI they're big enough to walk through, but only just, and are at shallow with about 3 feet of cover.

[x 7]

Re: It's just a small utility subway

I've just found an interesting report on the SubBrit site re the service tunnels, written in the 1890's. Also some photos - its clear that these are in places quite large and cover quite an area

http://www.subbrit.org.uk/sb-sites/sites/l/london_sewers/index.shtml

|The article also describes part of the sewer network

[x 7]

Re: It's just a small utility subway

this has a map of the service tunnels

second attempt at posting as the original link got truncated

http://bit.ly/1I8aO6N

[Tech Flack]

Our office lost power, which has been restored, all connectivity to our building, a Workspace shared office, is down. No timescale for a fix beyond that it will not be today. The Three mobile network I was teathered to winked out of existence early afternoon, so there still seems to be ongoing additional damage. Three expect a fix by Tuesday 7th.

[KBeee]

Every cloud has a silver lining

http://www.thedailymash.co.uk/news/environment/everyone-a-winner-in-holborn-fire-2015040297026

[Alastair Booker]

Looks like this has affected Verizon pretty badly.

[Alan Brown]

I've been wondering

When something like this would happen.

Central London's been running on overload for years and the undermaintained infrastructure blowing up more and more regularly as a result. It's like that scenario in Brazil min us trying to blame it on terrorists.

For an idea of how bad it can get, lookup the big Auckland nz power outage. Power was out to the central city there for 6 weeks...

[4ecks]

teh interwebs is borken

The "Magic Blue Smoke" leaked out in a big way.

[x 7]

We can make more - see http://uncyclopedia.wikia.com/wiki/Blue_smoke