Olympic cock-up knocks East London off Internet Businesses and customers in East London were still without internet and phone service this morning after Olympic contractors dug through a fibre optic cable. Contractors working on an Olympic venue near Stratford, east London, dug through the fibre optic cable on Saturday knocking a BT exchange at Old Street offline. One of …
Someone should tell the management at BT and at TI that diversity means more than just ensuring a decent ethnic balance. It also means running in two cables through separate routes into a building and having them travel in different directions. Reroute as required the traffic using extra capacity that you should have!
BT: F- must do better
When BT say the damaged stuff is in a deep tunnel, the guy who was doing the digging must have been really enthusiastic with the digging machinery.
The not-very-public rationale for putting BT cables in deep tunnels was that they were hard to break with bombs. Obviously, Mr. Bamford hadn't come to their attention.
BT state :-
"A large thrust borer being used by a third party construction contractor on work not related to BT‚s business penetrated one of our deep level tunnels in that area. The thrust borer has completely penetrated the tunnel and has caused material damage to both the copper and fibre cables in the tunnel.
BT engineers are working around the clock to restore service as soon as possible.
More than 70,000 PSTN customers were initially affected, though almost half of these have now been restored. Several thousand broadband lines, a large number of private circuits and Featurenet services have been impacted. Emergency and Police services were also affected but these services have been restored.
The tunnel is so badly damaged that it is not possible to work there pending structural support work and it is likely to be three days or more before that work can be completed and damage repair work undertaken. "
I go past Stratford train station every day and see the drilling equipment that they use - you are talking a hundred-foot-high, several feet diameter vertical boring drill, not some idiot with a Black & Decker.
Which makes it WORSE in my opinion, because you can't exactly NOT know where you're putting that thing. It looks like they are drilling to fill foundation columns for an extension of the station, in that particular case, but there's a lot of works around Stratford at the moment for the Olympics. However, my prime candidate would be that particular one ... maybe I'll shout out from the station later today and see if they point the finger.
Certainly having a huge boring machine chomp into your deep level tunnel is rare. But this seems to have taken out much of an exchange in the UK's capital city.
Are BT so cheap or poor as to not have fibre ring technology serving their major exchanges? The technology is decades old!
The BT report I saw indicated the tunnel was 32 *metres* below street level, but even 32 feet is well below the reach of most JCBs. There were 70,000 copper phone lines running through that tunnel as well as a lot of fibre links serving that exchange, some mobile phone masts and ISPs. For the copper wires, diversity isn't available (well, you can sometimes get a second line differently routed as a backup from the next nearest exchange, but obviously that's a very rare configuration) and for most of the others, it would be an extra cost option; it seems most of the companies involved chose not to pay the extra for full resilience. How many of us are actually willing to pay an extra 30% or 50% for our Internet access to average one hour of downtime instead of eight each year?
With an iPhone, if ADSL goes out I can still access e-mail and the web - and do so on the move as well; I did have a 3G datacard, but haven't used it for a while. This kind of outage might take both out for a while, but at least getting the mobile network back up will be quite a high priority.
From the photo I've seen, it looks like a pile-driver or something similar has gone straight through a tunnel perhaps 5-6 feet across. The tunnel is completely blocked by something painted yellow.
As this was 32 metres, or well over 100 feet below the surface, I really hope these contractors know where all the tube lines are around wherever they may be working or something much more tragic could happen. For sure there is going to be a big bill to be paid for by somebody. Quite apart from the damage itself, I imagine some outfits will want compensation.
They are doing massive amounts of work round there. They know perfectly well that there are all sorts of things underground. The line they hit was a major trunk, not an odd bit of cable.
How can they have not known where that line was, long before they started digging? Did they not do any checking, or did they manage to miss something that size during the check?
Maybe all the people and companies who lost service can get together and sue the diggers for incompetence.
I live in Dorset, but still lost my net access because of this. I'm with namesco/NDO who have 2 support lines. The premium rate one said that all systems were operating normally. The local rate one said that there was a problem with BT, and directed me to the website support page for updates.
Not a lot of point telling me to look up something on the web if I'm complaining that I can't see the web. Yes, I know that namesco have a dial-up 'backup' for their broadband customers, but all the details about it are on the website, which I can't see. Besides, who has a modem in their machine these days?
Thank you Reg for finally letting me know what's going on.
...the builders dug exactly where the planners told them to dig to avoid the documented location of all underground obstacles, but the documents were wrong? It's easy to point the finger of blame at the workmen on site now, but it wouldn't be the first time people have gone digging in an area that was supposedly clear of obstructions only to discover a tunnel/pipeline/etc that either was shown as being in a different location, or wasn't even shown in the first place.
What happened to Disaster recovery? As an (ex) IT Manager I ensured DR procedures were in place to limit effect on users. I can't get a job as I don't have the current NVQs, MCSE, ITIL etc,. I always felt doing the job was worth more to the company than attending training courses where a newbie tries to teach you how to do what you have been doing for 20 years.
So I am in Dorset; A fault occurs in East London and I lose my internet connection for over 48 hours?!
My son is around the corner, same ISP and he was unaffected. Probably the best quote was from Tiscali BB support when I was informed that I had a local BT fault. I suppose 140 miles is quite local to someone!
What happened to Tiscali/BT back up procedures that should have rerouted to alternative servers or routes.
Lets hope the UK ID DB is located in the same area and uses the same management teams.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
