Cable-cutting vandals disconnect Silicon Valley

Police are on the lookout for saboteurs believed to have gone on a fiber-optic cable slashing spree on Thursday, leaving hundreds of thousands of Silicon Valley residents without internet access, landline, and mobile phone service for much of the day. California Bay Area officials say vandals cut four cables belonging to local …

[Anonymous Coward]

Actually

It's overcast today here in Silicon Valley, so I have yet to see this mythical, fiery "sun" of which you write....

Alien icon, because the cable-cutting is just the first step in their fiendish plan to conquer the earth and to serve us all up a nice chianti. Keep your eyes on the skies!!!

[Anonymous Coward]

Cable-cutting vandals disconnect Silicon Valley

Let our freedomses ring! (c)

[Robert Simmons]

pfft.

Their Twitter feed has now announced the bounty is up to $250,000.

Dunno what they're complaining about though... 15 hours? Thats nothing, I was without broadband for 4 days thanks to those Olympic muppets ;-)

[Anonymous Coward]

Ok really

How many times do governments, public sector agencies, service providers, etc need to see these kinds of single point of failure incidents take place before they realize the old adage of "not putting all your eggs in one basket" holds even more truth now than it has in the past.

This is particularly interesting because now we know that emergency services across most if not all the silicon valley have no redundancy or fall over network in place. So all your local terrorist has to do is snip some fiber cables and bob's your fathers brother, you can run riot safe in the knowledge that it'll be quite a while before anyone is able to report what you're doing. Way to go guys /golf clap.

[Joe H.]

WHY?

If this was a vengeance thing, the perpetrator(s) are less than scum. If this was to cover some bank heist with the alarms down, then I'm still waiting for that story to break.

The union contract just expired so the vengeance angle is there, but, FFS, this is not the way to get a good contract from management.

Either way, I hope they catch them. Knocking out 911 service to thousands of people is stupid and mean, despite any reasons the morons had for doing this.

It might be expensive to do, but how about a centrally controlled magnetic lock/unlock system for these 100lb. manhole covers and or a silent alarm system?

The cops could be waiting at the top of the hole just as they were on their way back up.

"Pardon me but if you are not an authorized worker here I will have to arrest you now", said the officer, etc.

[Anonymous Coward]

Prybar - $15

Jumpsuit - $15

Bolt cutters - $15

Traffic cone - $15

Shutting Web2.0 down - priceless

[Richard Kay]

Well-planned attack

I used to work for a contractor on major trunk cables in the UK and have wondered what would happen if someone who knows the way the infrastructure works decided to attack it with the right tools in several places at once. Now we know. This isn't to do with single points of failure, because the attackers seemed to know that they would have to cut a number of cables in different places at the same time.

This is a difficult kind of attack to defend against, because these underground cables can be attacked almost anywhere along their route. I guess the AT&T staff and contractors will all be asked to watch out for any vans they don't know and report number plates wherever people are lifting manhole covers whether appearing properly equipped or not. Keeping a database of contractors vehicles and getting members of the public to phone license plate numbers in whenever they see a manhole cover being lifted could increase the risk of arrest for those involved in this kind of thing.

I've been told in some African countries telecoms can't involve underground line plant because thieves rip out the cables and sell these for scrap.

[Glenn Amspaugh]

Time for a resilient net?

The SyFy author David Brin has the idea of setting up all cell phones so that , when regular cell tower service goes down, they can be used to pass basic txt message packets, from phone to phone, as a way of allowing communication in areas that have their infrastructure disrupted (New Orleans/Katrina).

[Disco-Legend-Zeke]

Mesh Networks

The ARPANET was designed to be resistant to single point attacks of this nature.

Creating a neighborhood mesh fabric with wireless, copper, and fiber strands just makes sense.

Unless cuts were strategically planned to interrupt two points on a sonet loop, well designed fiber services should have survived.

[John Savard]

At Least It Wasn't Easy

Apparently heavy manhole covers do not provide enough protection for such critical infrastructure.

And cables were cut in four different locations, which could mean that the network did contain a degree of redundancy.

While Internet service is mostly used for casual purposes, telephone and emergency communications need to be better protected, and it should be possible to achieve this at relatively modest expense given the lower-bandwidth nature of the links. I trust the relevant authorities will treat this as a wake-up call, and act swiftly to ensure it cannot happen again.

Because next time, it could really be terrorists.

[Mark Pipes]

diversity?

In the old days of copper, there was path diversity. More than one route from point a to point b. This seems to be no longer the case. One thing that people forget is that the cell phones are always the first to go down. They need to insure path diversity, and media redundancy for critical systems (911, etc). Perhape back up fiber with microwave, for example. In this case, it was evildoers that cut the lines, but earthquakes can be just as effective. Having a redundant line in the same trench does you no good when the whole thing snaps. Whoever engineered away the diversity should be prosecuted as well.

[jake]

@John Savard

First: John, the following is not intended to be picking a fight with you ... Yours was just a convenient foil for a few points I wanted to make. Remember, I grew up in Silly Con Valley, I work in the industry, and I know what the infrastructure & politics look like from the trenches. Or holes in the ground, if you prefer.

"Apparently heavy manhole covers do not provide enough protection for such critical infrastructure."

They never have. I mean, they are only 100 lbs. That ain't heavy. A crowbar & a hacksaw & 3-4 minutes is all a reasonably fit human needs to commit this particular crime.

"And cables were cut in four different locations, which could mean that the network did contain a degree of redundancy."

A large degree of redundancy, at least for the most part. Most of the areas that went down were (by my estimation) semi-rural, with limited or no redundancy. SLV and Santa Cruz, especially.

"While Internet service is mostly used for casual purposes"

Mostly? Probably better than 98.5% ... Our Percentage[tm] (STR)

"telephone and emergency communications need to be better protected"

Agreed. 100%.

"and it should be possible to achieve this at relatively modest expense given the lower-bandwidth nature of the links."

Actually, the folks who need it have their own radio systems for emergencies. The problem here was that the public wire going into dispatch was down. In other words, the emergency folks couldn't get calls from the public. That's a difficult, but not impossible, nut to crack.

As a side note, what's the aggregate bandwidth for all the telephone calls happening in Silly Con Valley at this moment? How about just the calls between Santa Clara County & Sacramento? The numbers might surprise you.

"I trust the relevant authorities will treat this as a wake-up call, and act swiftly to ensure it cannot happen again."

I doubt it. It's been discussed on RISKS Digest on many occasions (Metacrawl comp.risks for the archives) ... And I personally have discussed the issue with City, County & State people here in California. Nobody who has any power has any idea of the issues involved. Management is not interested in RealLife[tm] security that will cost a bunch of money to fix, unless it's a lot more photogenic than a bunch of street-level padlocks.

"Because next time, it could really be terrorists."

There aren't any. So-called "terrorists" are weapons of mass distraction. (Queue amfM ...)

[Anonymous Coward]

Satelite links?

Is it just me, or is it really true, that these morons from AT&T have no temp satelite uplinks?

it is os easy and so simple to do and setup.

of course the network would be heavily congested, but it would still allow traffic.

I would say either terrorism isn´t as bad as our politicians lead us to believe, or the guys from AT&T have excessively bad and careless network architects

[Frank]

@Disco-Legend-Zeke re. Mesh Networks

A *national* mesh fabric would be even better, with sub-level neighbourhood meshes too. You could follow that with data prioritisation protocols etc (nobody would mind losing the ability to watch You Tube if it meant that emergency services calls could get through).

All this can be done and the network/data/routing techniques are known. However, to do 'what should be done' will need planning, investment, integration, cooperation and coordination between various levels of national government, local government and a mix of telecomms companies. So, cynical as I am, I don't think it will happen. Nobody died (we hope) as a result of this sabotage and service is now back to normal so most people will forget about it until the next time and the next time.

[Anonymous Coward]

@comments about single point of failure.....

You didn't read the article did you? Several cables were cut, you can only do so much to provide a resilient network. With 4 cables being severed it seems like there were 2 rings taken out, for a metro fibre network that's pretty much it. Looks like the attakers knew the ring architecure and where to cut to cause maximum damage.

[gollux]

Specialized Tools needed...

to take down fiberoptic communications. A crowbar and a hacksaw. And that's all it would take to completely knock the southern half of my state off the telecommunications map.

[Anonymous Coward]

Thames Water

How long have they been working in California?

[chris]

....nope.......

....can't bring myself to do anything but laugh.

[Seán]

Great

Now maybe some slackjawed IT managers can be fired for having no backup comms links. Just how amateur can they be.

Also any tool who actually uses SyFy should be killed FYI.

[kain preacher]

@Iam Me

Umm if you are not a local I can see how you can think that silicon valley might be vulnerable but you are wrong. Let me explain. They area that was effect was south county aka southern Santa Clara County. Now to get to there you have to drive down 101. Going down 101 there will be ten minutes of nothing out there , Just wilderness. The link was severed in area that was rural. Another ten minute drive and you are in farm country . Now 45 minutes to north were the cable was cut in two section nothing happened because it was in a urban area.

The message is cut a fiber link in a urban area nothing much mate . Cut a link in a rural. area you are screwed .

Ps no in that part of California would consider it silicon valley .

[James]

Russia/China/Korea/Iran

It would be a safe bet that China and or Russia and possibly Korea and Iran (though probably not) know the locations of these links and spend a lot of resources making sure that they know the locations of them. It would probably also be a safe bet that the US/Europe knows the locations of the others links.

In the event of Wars on Stuff these would be high priority (and effective) targets. Pity you can't really defend them as they snake through farmers fields and the like...

[Alan W. Rateliff, II]

Turn myself in...

For $250,000, I would turn myself in. Sure, I would have to explain how I got across the country and back in such short time, how I became savvy enough of the network in the area to disrupt it (I could blame that on Google Earth and Street View, as seems to be chic to do these days,) how my frail body was able to life the man hole covers, and so on.

Sure, there would be a trial, fines, and jail time, but $250,000 reward (yeah, would have to get a buddy to turn me in and split it,) then sales of the rights to the book and movie. Man, I would be set for life!

Paris, selling all rights.

[Charles]

@Iam Me

Redundancy won't work against a planned attack, either. It may take more effort, but you can be sure anyone intending to cut off emergency services will simply make the effort to cut off ALL redundancies. Note that this one incident had five separate cable cuts. Those five cuts could've easily been done to address any known redundancies. As for using different technologies for the redundancies, each one has weak points that could still be simultaneously struck.

[Anonymous Coward]

I thought the calibre of the net

had improved, then Silicon Valley came back.

[gollux]

Fiberoptics and redundant rural communications

For everyone out there talking about single point of failure and redundant paths. Rural areas are lucky to have one single cable bringing them service. If they have two, the cables will have nearly parallel routes, especially in mountainous areas.

Think of it this way, going up a box canyon, you probably would have cables running on both sides of the road, if you have some sort of military or commercial installation that rates the extra expense. They aren't going to run a cable cross country over the 2500 ft ridges that surround the access to your community unless that's the only way to get there.

There isn't enough satellite communications capacity to give you redundant communications for all these outlying areas, and for the one or two times a decade it's needed, the equipment expense and personnel to run it are already not going to be in an already bankrupt budget in a bankrupt state.

A microwave link backup across mountain tops might be your answer, but these have been being phased out as requiring too much upkeep in our area. Other types of radio interlink get congested extremely quickly and just don't have the carrying capacity we've come to expect as provided by fiber optics.

Us rural people already realize that the convenience of the 911 call doesn't really apply to us anyway. With emergency repsonse 30-60 minutes away, most things urban dwellers take for granted are only a far away dream.

[Anonymous Coward]

@jake

Sure seems like an inside job. Some "smart" employee that's not gruntled enough. (Keep your employees gruntled!) That's why locks on the manhole covers won't help - insiders have (or can get) keys. Apparently this happened before, also just when the CWA contract ran out...

[Matthew]

you would think..

that emergency services at least your have some form of backup such as a satellite uplink... not that their would me much to do if no one could call 911

[jake]

@AC 20:27

"Sure seems like an inside job. Some "smart" employee that's not gruntled enough. (Keep your employees gruntled!) That's why locks on the manhole covers won't help - insiders have (or can get) keys. Apparently this happened before, also just when the CWA contract ran out..."

I'm not going to speculate on whodunnit. Outside my job description. But my employees are quite gruntled, thank you ...

Locks will help if it's J. Random Criminal out to cause a bit of trouble, or a couple highschool kids having a lark. They probably won't help if it's an organized effort, with insider information, but in that case you're screwed six ways from sunday anyway. They also won't help if the perp knows how to pick locks, which isn't exactly rocket science. Locks only stop casual crime, which I suspect this was, thus the comment.

I have no idea about the CWA contract. Unions are anachronistic and just as bad as the management that they supposedly protect the workers from. Don't believe me, union card holder? Take a simple test: What are you driving? What's your Union Boss driving? What is Management driving? Q.E.D.

[Ru]

Resiliency?

The ARPANET wasn't designed to be resilient to having multiple connections severed in what looks suspiciously like a well planned and organised attack, not merely vandalism. It was intended to allow communications to be routed over a less than reliable network, and able to survive the drop-out of a node or two.

There really isn't an awful lot that can be done in this situation... adding further connectivity in the form of microwave transmitters (expensive, much slower than fibre) or satellite uplink (exceedingly expensive, really, really slow) probably doesn't make economic sense, or it would have been done already. It will presumably require legislation or subsidy to encourage the installation of such things.

Mesh communication technologies already exist in the form of trunked packet radio systems like TETRA in Europe (though I don't know what US emergency service or military use). They're complex, resulting in bigger, heavier and more expensive handsets than the mobiles we're all used to... emergency service radios also use higher power transmit levels, so battery life is much lower than a GSM phone, and scaremongering cancer stories will be an order of magnitude more irritating. Given that the consumer will not like any of these flaws, and 99.97% of the time they will not use or notice the benefits, who is going to buy?

Trying to extend this to produce community- or county- or nation-wide mesh data networks seems to be doomed to failure. Too complex, too slow, too uninteresting to anyone who isn't a network or radio nerd. Once again, nothing short of subsidy or legislation is going to help here.

Having said all that, Iridium satphone contracts for emergency services wouldn't go amiss.

[kain preacher]

@CWA

Its a weak union. If they go on strike no other union will honor. No if you said it was the IBEW , yeah I really could see some of them cutting those lines .

[Anonymous Coward]

Recession era job...

After having been laid off back in late 2008, I recently went to work for AT&T in early 2009. I was immediately forced into the union, and under the old (now-expired) contract all new hires don't get health care for six months. AT&T also misrepresented the position as a "network center technician". What a load of bull. I work in a call center. The union employees are treated like dirt (read this as wage slaves, or just actual slaves), with AT&T requiring nine hours of mandatory overtime nearly every week or you get fired. Did I mention the pay sucks as well. AT&T has to be the worst place I have ever worked and I'll be quitting the moment something else comes along. Knowing what I now know about the phone company, I can only lament that when they broke up AT&T, they didn't shut it down forever and start from scratch.

If the perpetrators were AT&T union employees then I have to give them a lot of credit. They demonstrated far more testicular fortitude than the union reps negotiating the new contract. IMHO, given the current working conditions, the union should have gone out on strike the moment the contract expired, regardless of the fact the US is currently mired in a recession.

[Anonymous Coward]

Kind of rural

While silicon valley proper didn't seem to be affected as far as I heard, here in Santa Cruz (which I personally don't consider rural) and Scotts Valley (location of Seagate HQ) there was no internet (apart from comcast!), no landlines, no leased lines (T1), no DSL and no cell phone reception. These two locations alone probably make the number of people affected by the outage the same size as the the east london cable snafu - except I didn't see anything about cell phone reception being impared in London.

Cell phone reception and T1 came back around 2pm, with DSL around 8pm.

Due to the geography of this area, I am not surprised that all connectivity comes from the direction of San Jose.

[Anonymous Coward]

@Joe H.

this is not the way to get a good contract from management.

The folks who do stuff like this aren't thinking about the contract. They are just PO'd that whatever changes are coming are most likely going to hit them in the pocket, and fatten the wallet in the top management tiers. It's not uncommon for this to happen every contract renewal time. I don't speak for management or union. Heck, as my wife; I don't even speak for myself.

Personally, i would make sure (if I were ever stupid enough to do something like this) that I had no accomplices or anyone who knew. Gitmo is still open for business.

[Anonymous Coward]

@Joe H

"The union contract just expired so the vengeance angle is there, but, FFS, this is not the way to get a good contract from management."

Of course not - everyone knows that it's much more effective to hold executives hostage!

[Disco-Legend-Zeke]

Mesh Networks II

Actually, my original thinking on mesh networks was for Internet-3.0 (sm) a packet switched, wireless network that would be a successor to 802.11n.

Of course much of that exists. the real innovation would be using GPS location as a routing tool.

A network of this nature could not be taken down by single or even multiple failures, packets would just route around the big-hole-where-your-block-used-to-be.

The biggest problem with such a system is that there is no big money to be made by carriers, they are now BUYING our spectrum to prevent this from happening.

Americans may want to add their comments to http://www.adtran.com/pub/Library/Broadband%20Stimulus%20Articles/March%202009/FCC%2009-31_%20A_National_Broadband_Plan_For_Our_Future-%20NOI_4-8-09.pdf (that is a truly ugly URL, sorry)

Just in case you think this is 3,14159... in the sky, DARPA has already developed a military version of exactly this. Radios thrown out randomly to provide a mesh path.

paris cause i miss her.

[John Werner]

Emergency Service Backup Comms

There are systems for backup comms for Emergency Services, and they aren't even commercial systems -- they're volunteers using their Ham Radios. In the US, the Amateur Radio Service is chartered to provide a core of trained radio operators for emergencies.

According to this report in the Mercury News (http://www.mercurynews.com/centralcoast/ci_12115324), :Ham radio operators organized themselves to assist with communications between hospitals. About 25 operators were handling the radio traffic while another 50-60 had signed up to fill in when other volunteers got tired, operator Craig Smith said."

- John

[Anonymous Coward]

Fire Sale! Nah....

Didn´t anybody even consider the pranksters/ nutjobs /whatever were searching for copper cables to steal it? You don´t need to be on some remote African country to do it, this happens A LOT here in South America too, just to honor the only comment which considered the possibility.

As far as I know even Berkeley University suffered an outage some 3 years ago when some fiber cables were cut. Apparently, the insulation for both is exactly the same, and anybody but a contractor with the blueprints of the cable paths knows which one is which. They solved the problem with a warning such as "Optical fibres only, no commercial value" at every manhole or 200 meters along the path of the most often vandalized lines (I don't know each one). The vandalism stopped completely, since there were nothing to steal (and never was) anymore.

Here in South America they turn to suburban train power cables. Now, these are pure copper, carrying 3000V CC energy, what should make it harder to steal. They throw a thick steel cable on the lines, shorting it out on the rails, to disarm the circ. breakers, in order to steal it, according to police report. The rail company got smart to it, set a special alarm to disarming circuit breakers, set a trap with local PD, and caught an entire gang of "preemptive recyclers of copper wire" red-handed (pun intended) with a truckload of copper thick-gauge cable, used to power the trams. The cables are easy to identify, since they are branded "Railway Co." or such every 20 meters...

See, it may not be a Fire Sale, it may be just ordinary copper wire robbers.

The icon of choice seems so appropriate...

[jake]

@AC 12:35

There is no such thing as a "fire sale". It's a term made up for a rather silly movie.

The fiber & power cables don't look the same. They aren't marked the same, either.

Whoever did it knew exactly what they were doing. Why is a matter of speculation.

My gut feeling is that the San Carlos perps were copy-cats ... Not sure why, just a hunch based on the televised cut marks on the cables.