Midnight theft left Vodafone users bereft Vodafone's Monday morning outage was caused by thieves who broke into the operator's Basingstoke exchange and lifted a load of switches. The break-in happened around 00.30 this morning, and the police were quickly notified. Vodafone noticed its own network collapsing and assembled its "War Room" which is supposed to deal with …
None of the users here could get a signal first thing, and we are about 1 mile from the Atantic ocean - hardly "South East England".
As for "only 100,000 people affected" - pish, tosh and other words. There were probably that many complaing on the various forums, but that is not the sum total of those affected.
Mind you, I love the comment on Twitter by someone that tried to cal someone in the Vodafone PR office, only to hear the message "Welcome to the Orange Answer phone service".
The architecture of the "intelligent" networks and its IMS based 3GPP successors has a considerable number of elements which are very difficult to have on hot-stand by and/or in failover+load-balancing mode.
This is something inherent to the arch in question. If you take out a BSC or RNC everything slaved off that BSC is dead and you have to reprovision a significant chunk of your transmission as well as commission a whole new BSC to fix it. If you take out an MSC the story repeats just multiplied by a factor of several thousand times for all of those BSCs or RNCs that have been orphaned by the death of their master. If you take out the policy function, the diameter or some of the key bits of the EPC a big chunk of the network dies in a similar fashion.
In fact it is inherent to the idea of the intelligent network. The more intelligent the network the more vulnerable it becomes to someone taking a big axe and surgically removing key bits of the "intelligence".
"In fact it is inherent to the idea of the intelligent network. The more intelligent the network the more vulnerable it becomes to someone taking a big axe and surgically removing key bits of the "intelligence".
Technical issues aside, Vodaphone sell a business service, they should be planning on bits failing, or clearly stating that their service is not suitable for mission critical work.
If they want the business service profits, then they need to pay up for the backup coverage
Definitely not fixed here in Guildford. I can make calls but anyone calling me is helpfully told that 'the number you have dialled is not recognised'
Data is extremely sporadic too, I had a bunch of emails appear on my Blackberry about an hour ago but the thing has been silent since.
Yeah, took out our entire company, and everyone I know with vodafone - how exactly do they work it out at only affecting a few customers?
Service back 10 minutes ago, no calls/txt before then.
Vodafone clearly has massive single point of failures in their network, and very poor backup planning!
Hi,
14:45 still unstable around Reading.
This poses serious question about disaster recovery plan at Vodaphone... If the Basingstoke datacenter was destoyed by fire it seems it would bring south (eastish) UK network down.. for how long?
I suspect other key premises are in the same situation.
Any SLA / compensation planned?
...that'll be 50p compensation for service lost for half a day.
Frustrating as outages are, I don't think Vodafone contractually owe retail customers more than that (and even then, it's a stretch, as they cover themselves with protective clauses in their T&Cs).
Everything gets cheaper, the only thing we care about is how much we get nobody ever stops to question the quality, what backup is there, what resilience.
I don't imagine any other carrier is any different.
They may review their security procedures but that is it - nobody really cares and everybody will have forgotten about this when the contract comes round again, they'll care more about which network has the shiniest phone and the biggest data package / most texts.
Corporate seriousness chart.
1. Bonuses.
2. Wage level.
3. Company car.
4. Perks.
5. Executive drinks cabinet security.
.
.
134. Infrastructure security.
135. Customer satisfaction.
As a back-up, get a POTS phone that doesn't need mains or a wireless connection. No frills. Under a tenner.
The simpler stuff is, the more reliable it is.
Never trust or rely on technology. Never trust or rely on those who provide technology or tech services. Alway back-up your data. Always have a plan B for when the tech fails (which it will).
The use of POTS as a back-up is indeed a good plan, however there seems to be a move away from providing a simple piece of "wet string" in new buildings. As the IT guys seem to be responsible more and more for cable infrastructure they like Cat5, hubs and switchers. At a recent job in Turkey seven new sporting venues had been built without any from of POTS infrastructure.
Back to Vodafone, I hope they learn the lessons from this incident. They have provided me with a good service for over ten years now and, in my book, still have the best coverage.
Not sure how their network joins together and it might be a red herring, but I'm about 12 miles from Basingstoke and have enjoyed 5-bars uninterrupted service all day both home and at work. All of my colleagues are offline. The only difference is - I have a Voda SureSignal box at home & work?
Coincidence?
Suresignal comes out of the network 'up north' somewhere. That is why I had great reception this morning at home and got my own network wakeup call at 4:28 when some of our services went down. But unfortunately once in the car and into London it was gone.
It is slowly getting back now but still very patchy despite my phone showing good reception.
http://forum.vodafone.co.uk/t5/Mobile-Broadband/Vodafone-Still-Stalking-Subscribers/td-p/671569
Somebody complains about illegal interception and then suddenly their exchange gets broken into? Coincidence or somebody trying to get rid of evidence? (where's the black helicopter icon when you need it?)
Not everyone is using their mobile to play FarmVille on their mobile. Everyone in our office actively trades with people on the road who rely on their mobiles to earn their (often self employed) living. While it hasn't affected us directly, it's been bloody annoying not being able to get hold of many of our contacts.
It's guaranteed to have cost a good few people money. Sure, they're only paying Vodafone £1 per day, but it's more than a little embarrassing for Vodafone to demonstrate how bad their disaster recover is.
As somebody else has commented, you have to make cutbacks to generate £11.5 billion profits. Obviously security was one of those areas.
This post has been deleted by its author
I have been using Voda fone for the last two years for my phone and mobile broadband connection , I have always found their service excellent .I think a bit of commercial espionage could be behind this attack and it's not the first time this has happed.When my service went down today I rang 191 and went straight through to their bonnie call center in Scotland and as usual spoke to a polite and competent person who gave me the full details of the situation and a free £5 top-up to cover my inconvenience.When I was with T-Mobile I had to talk to some toothless old git in a tent in the middle of Gobi desert reading from a script in badly broken English.
We have three VF handsets here. Only my tocco lite would rcv a call for a short while but no handsets would originate a call. Then an hour later ~3:30 incoming died and the balance OSD died as well. test calls at 6pm and the VF handsets are still DIW
I suspect they are still rebooting and recovering from thier outage.
My worry was in checking our PAYG balances to ensure that afterthe outage we have not lost existing credit (again!).
My last fiasco with VF was a android handset eating 1UKP/hr when switched off and battery removed. which is why I moved to gifffaff for the android handset.
Now I understand why others have more than one handset or a dual sim card handset.
Our dispatch system uses GPRS to communicate bothways. It disappeared at about 7 am for about 7 hours. unfortunately the only back up is mobile phones! That's about 40 self employed people who can't earn.
Whoever the Vodafone DR person is, they must be feeling somewhat insecure at this point. and justifiably so.
however I am available should they decide to revisit the plans..
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
