Where was the Data Centre?
In the middle of a field? I had the mistaken belief that they were holed up in basements about 3 floors underground.
Did someone leave a window open?
Millions of Vodafone customers were disconnected this morning after an overnight break-in. Calls, text and data are intermittent west of London with a fix in progress. The failures seem only to be affecting areas just west of London, in the so-called Silicon Corridor. Wales seems fine and the rest of the country is OK, but …
Intermittent my Arse... I got a "curtsy call" at 04:15 from ADT to say the alarms PAC (GPS) signal was down, that is obviously when the scroats kicked the door in.
I now have an entire work force with no mobile communications. Does the phrase "single point of failure" spring to mind.
<quote>
I now have an entire work force with no mobile communications. Does the phrase "single point of failure" spring to mind
</quote>
Yes, it does... why would a company so clearly reliant on mobile comms have all their staff using a single mobile network??? Surely a mixture of vodafone, o2, t-mobile, orange & 3 would be more appropriate for your company??
When you point a finger at someone, there are 3 pointing back at yourself!
Fraid I have to agree with everyone else - running something that important on one network with no apparent back up plan? Perhaps you can use today's downtime to consider how much you'd not lose by having one.
Suggestions: T and Orange allow cross network roaming on 2G; have a draw full of PAYG sims on various networks for next time and/or network coverage evaluation.
Yes it was funny!
<quote>
I now have an entire work force with no mobile communications. Does the phrase "single point of failure" spring to mind
</quote>
But
<quote>
Yes, it does... why would a company so clearly reliant on mobile comms have all their staff using a single mobile network??? Surely a mixture of vodafone, o2, t-mobile, orange & 3 would be more appropriate for your company??
</quote>
A mixture wouldn't work! and here is why.
If a large company uses mobile Telecoms, they will stick to one provider because they need to keep costs down, having multiple contracts with different providers wouldn't work as the company would not benefit from free calls to the same network or the (friends and family) business type perks.
I'm surprised people are criticising this guy over his SPoF remark. Is a business that procures a load of phones for their employees going to get a mixture from different service providers? Come on, they'll be looking for a bulk purchase deal, standardisation and saving hassle for their accounts department which would have to manage payments to multiple providers. Does your company have different landlines from different providers - no, I didn't think so.
Anyway, I'd be inclined to agree with the OP - you'd expect Voda's systems to have various load balancing & backup systems that should prevent a meltdown like this over such a large customer area. And where was security? Surely, a major telecoms IT centre would have 24-hour security??
So i should not use just tcp/ip on the LAN because its a single topology,
I shouldn't use one manufacture of Hardware even though they are biggest.
Perhaps I should look at getting the coffee from three different sources in case one stops brewing
Either you work for VodaFone or a government establishment where cost is not a concern. We use VodaFone, the biggest supplier of mobile networks in the country, in order to minimize the ongoing admin and management costs of maintaining a fleet of phones.,
I quite understand that you use one network to control the billing, the cost would be horrific if you split over multiple networks however...
How much exactly would it have cost you to call O2 and ask for a set of backup PAYG SIM cards for your staff and issue them, along with a secondary phone number list? That way when Vodamuppets bugger up their network you all switch over to the secondary network, given that the problems are rarely complete failures many SMS would get through instructing staff to switch to the backup SIMs. You can always use a web page as a secondary notification.
I quite understand your situation, we have been without a fixed line Internet connection for over two months thanks to the time wasting lazy tosspots at British Terrorcom, good job we have a Vodafone dongle and a spare router which lets us provide (albeit slower) Internet access to our network for key services such as email.
Several replied to you, but here would be my response: Actually, I work for schools. You know, those things with zero IT budget and what is there is spent on Visual-Basic-filled-with-clipart "curriclum-vital" software.
You're complaining about having a single point of failure when you use a single company. If you have Cisco-only gear and Cisco go out of business - you're stuffed. You can say "it won't happen" or "we need to use Cisco" until you're blue in the face. When it goes bust, nobody will care about your problems.
If an exploit hits a Cisco-only bug (say, the million-and-one IOS bugs), you're stuffed. I'm not saying you shouldn't use Cisco, but you should be prepared for the consequences of such a mono-culture. It's like saying "All Windows PC's are more susceptible to viruses". Completely correct but sometimes you "have to" use Windows. It just means: Don't be surprised when a virus hits and all your Windows stations go down (but not your Linux stations), or all your Cisco IOS gear is exploited (but not your D-Link gear or whatever), or all your engineers are out of reach because all use one mobile phone network. Complaining that MS, Cisco or Vodafone are to blame for not having their own processes to manage their mono-cultures is as silly as you relying on that mono-culture. It may be *practical*, *sensible*, cost-effective, but it's still a mono-culture.
And thus, you can't push blame to Vodafone for not doing your job. Yeah, you can complain and get service requests and claim on your business support contract (you *DO* have a written legal statement of constant availability from Vodafone, yes?), but nobody can "fix" that problem of you only using one operator because "they are the best" (this incident kinda negates that kind of thinking, I believe). Any one of your people could be using any mixture of any mobile operator without any problems at all and no difference to you except three separate bills instead of one combined. Or, in an emergency, any of them could be instructed to get / issued with a PAYG SIM on another network for the day.
My school has a dual ADSL2+ line load-balance/failover setup - we get 48MBps of Internet connectivity and can turn off any one modem without affecting the school in any way (who rely on the Internet to fill in exam papers, pay wages, contact parents, etc.). Both lines are linked to our local exchange. We *know* this and don't expect it to operate in the case of an exchange fault - it's to cover *other* eventualities, or else we'd get in a leased line from another exchange deliberately.
By a similar token, you should NOT expect your business to operate in the case of a mobile operator fault (or even the tower nearest your main site of operations having a fault), by the sound of it. Tell me, what happens if your VOIP/landline telephone system goes haywire, or are you just relying on that to "always work" too? Our actual "emergency" backup for Internet connections is a handful of 3G sticks all on different mobile operators that can run the entire school network just by us plugging them in and running a script (they are always active because we also use them for SMS reception / transmission). Latency goes up, but users continue as if nothing had happened at all. When we hit the traffic limit on one stick, we replace the SIM with some other PAYG with £1/day data. Not perfect, but it works and gets us out of trouble.
Where's *your* emergency backup, except to shout at Vodafone and say "it should work, it's been working for years!"?
And personally, we do *not* rely on one manufacturer for PC's, printers, etc. It's just too easy to have a problem with a certain support contract, manufacturer, buggy driver, etc. and get put into an impossible-to-fix-in-time situation. Even our two IT suites (13 computers each, so you can tell we're HUGE) have 13 identical computers but of two different models for that reason. There is a hot spare of each type in my office. Staff laptops are many and varied, for that reason. Printers are many and varied, for that reason (some manufacturers like to sting you on toner / drums if that printer wasn't very popular). Every server is a different manufacturer. We deliberately mix manufacturers in RAID array drives where possible. We even have two unique lines and two different types of fax modems connect to the main Linux server (which has a "hot backup" already imaged and sitting in the next room in the case of a problem) to send / receive faxes over the network.
Redundancy and avoiding monoculture isn't hard. It doesn't mean it will save your arse or cost the world. But saying "what if" to yourself about business-critical things occasionally is always a good idea. Ignore the whole thing about manufacturers and suppliers. Now just say to yourself "what if the phones don't work" and then think of several reasons why that might happen (local tower fails, landline fails, extended power failure in the local area, 9/11-like-incident swamping the phone lines so they are limited to emergency use only, etc.). You don't need to be IBM to be thinking about things like that. It was important enough for you to rant about how it's all Vodafone's fault. But not important enough for you to have ANY backup whatsoever, even in an emergency?
Slackware 13.1. Has custom kernel with load-balance routing patches from http://www.ssi.bg/~ja/#routes
- Install two (or more) ADSL2+ modems on two Ethernet ports, configured to just push all config to the Ethernet port ("Bridge" mode or whatever your device wants to call it, basically DHCP on the Ethernet cable should prompt the modem to ask for an IP from your ISP and propogate THAT IP and all packets back down the Ethernet cable transparently).
- Install one network card for LAN (Gigabit with Squid proxy and 100Gb disk cache for our setup).
- Do the usual NAT / iptables magic (nothing special at all, haven't modified a single one of my usual firewall / NAT scripts)
- Put in a default route as mentioned on the page for those patches that "nexthop" to the two external Ethernet cards and make sure it's in your startup.
With the patches, you can pull a plug, and everything just continues on as normal, even when you put it back in. Without the patches, your routes disappear and you're into custom scripts to detect failed connections and rebuild routes each time - not so tricky with DHCP but still a royal pain.
And extends to any number of interfaces that present themselves as a network device - 3G stick, PPP Modems, etc. The routing just handles it all.
Only problem is remote access / hosting, which you have to take special account of. Otherwise the machine "chooses" a favoured route for each path and if your remote users don't connect to the right IP that gives that route, you get nothing. Of course, you shouldn't be hosting anything important on an ADSL connection, and I personally give each VPN user *both* IP's and tell them to try them both (it is failover, after all, so one may well be down), others have an iptables / routing exception hard-coded to always accept VPN traffic on only a single given interface.
As a side bonus, you also get a lot more torrent-peers by having two connections....
You pays your money and makes you're choice!
I doubt it was your choice to make, more like a dictat from on high but still the same thing applies. The management tell us to cut corners, then when it comes back to bite them in the arse, we all get to carry the can. I love working for a living!
My company provides me with a Vodafone handset for general use, but also an O2 handset for backups (less features, really just for voice calls). If both those fail, then I use my own Orange handset and bill them for expenses. Failing everything else, I have memorised the number for our Fault Control, and will beg, steal or borrow a phone. That also assumes that all the signal post telephones have also failed and I cannot even communicate through a signaller (yes, i work on the railway). So, I have three methods of comms provided by my employer and at least two of my own.
Maybe you should look at your own resilience before blaming everyone else. I agree Voda should have a bit more security and redundancy in their data centres, but your company can't be much better.
BUT...
Vodafone Group plc (LSE: VOD, NASDAQ: VOD) is a global telecommunications company headquartered in London, United Kingdom.[2] It is the world's largest mobile telecommunications company measured by revenues and the world's second-largest measured by subscribers (behind China Mobile), with around 332 million proportionate subscribers as of 30 September 2010
The Reg did specifically state by profit/revenue not by UK subscriptions... Why don't you be a good little boy read the article and stay off the smack?
thanks.
"in order to minimize the ongoing admin and management costs"...
There's your problem. Minimal cost rarely equals maximal service. If mobile comms is business critical for you, choose the option that allows you to run your business in all foreseeable situations, not the one that costs the least.
Two ways to look at it:
First and foremost, once upon a time Ofcom (which was called different then) and the mobile operators instituted the no-in-country roaming policy. In those days nobody looked at the mobile phone as a possible single point of contact. That time has clearly passed and having at least mvnos with multi-operator roaming is something that is long overdue. There are businesses that need that level of connectivity nowdays.
Second, you could have been prepared for it yourself (at least to some extent). For example, I do not give away my mobile (for a couple of years now) and I have a PBX which tries based on time of day, schedule information from my calendar and a few other things my fixed line(s), voip and falls back to a list of mobiles (including foreign ones) when needed. Once the list is exhausted it goes to voicemail which I get as an email as well. It takes me about 1 minute to add or remove a mobile or VOIP endpoint from that list. If you had that, you could have just told your work-force to go and buy a throw-away pay-as-you go from three, O2, T-orange or whoever else you fancy and get on with business as usual. In fact you could have told that to them on their _WORK_ phones and left them to pick up the voicemail via whatever means they find, whenever they get to it.
"I now have an entire work force with no mobile communications. Does the phrase "single point of failure" spring to mind."
Comment from a friend of mine: "At some point last night, Voda realised they should have issued O2 or Orange mobiles to their callout engineers..."
Another nice slice of irony in our (rather major) Vodaphone-only company, though - we recently opened a new major office, where it was rapidly discovered only the Vodafone network got any decent signal in most of the building, despite perfect service levels on all networks immediately outside. Accusations of network blocking were dismissed with a 'not our problem - you can always change networks' from the brass.
Now the office has been beautifully devoid of annoying ringtones all morning, and the poor little Blackberry addicted dears are showing serious signs of withdrawal symptoms...
"So i should not use just tcp/ip on the LAN because its a single topology"
TCP/IP is not a topology.
"I shouldn't use one manufacture of Hardware even though they are biggest."
That's right. Use multiple vendors. Don't use something just because it's the biggest, that's a stupid reason to do anything.
"Perhaps I should look at getting the coffee from three different sources in case one stops brewing"
Or just could stockpile coffee and change when you run out? Hardly a comparable analogy, is it?
We don't know the motives for the break in, yet, but Vodafone has been the named target of a number of anti-austerity groups like 'UK Uncut' and Socialist Worker. The company's alleged 6 billion pound tax evasion, inan age when ordinary folk are being expected to endure a reduction in lifestyles, is the stated reason for this. A Vodafone shop in Nottingham was occupied recently by UK Uncut, and there have been vague threats from other groups, to cause deliberate damage to Vodafone's business.
It's one thing to have a backup plan about thieves breaking in and accidentally stepping on a few cables while they made of with a server; quite another, to plan for a deliberate act of vandalism.
Like I say, it's just speculation, but I wouldn't regard a building full of servers, as being the typical sort of place to rob.
People seeing problems in Cornwall/Devon also.
I agree with comments regarding security ... ^ massive fail Vodafone. Single point of failure?
Internal mail:
--------------------
Vodafone Technology Incident Management
Service Operations
.
Incident Update
Source Remedy Reference
Incident
Service Affected 3G, 2G, SMS, Voice, Vodafone One Net - Voice, Paknet, Telephony, Vodafone.co.uk, BlackBerry Connect, Email, Network Mgt, Voicemail, Fixed Link (Voice), Remote Access Service, MMS, Severity P1
Status Open Comms ID:
Start 28/02/11 00:40:00
Incident Statement Link connectivity issues are being experienced to thte Basingstoke Data Centre
Impact Multiple services are impacted. Customers may experience problems making & receiving calls, sending and receiving text messages & establishing data sessions. War room invoked, conf call ongoing to discuss action plan. Crisis Mgmt engaged.
Last Update
Next Update 28/02/11 08:00:00
[Anyone have access to this system and fancy sharing any further updates, please do?]
"War room invoked" = Brooms and vacuum cleaner relocated to corridor.
"conf call ongoing to discuss action plan" = PR machine ramped up to "full spin". Arses being covered.
"action plan" = Headless chickens sighted in management suite. BOFH and PFY making coffee and filling in overtime claim.
"Crisis Mgmt engaged" - Press release being prepared.
It's the fluorescent yellow one with "Seecureity" on the back.
... given the responses to the SPOF comment above. But I agree with you. You would have thought any Enterprise system that is business critical would have resiliency built in.
Clearly Vodafone don't think their networks are business critical. I need to re-read my T&Cs to see how long the buggers have locked me in for. :-(
I managed to speak to one of the staff at the customer service desk around 9.00 am. He told me that it was a software fault, and that it affected a number of parts of the country, inlcuding South West, parts of the Midlands and North as well. I'm not sure if he even knew anything about the breakin.
I did suggest that perhaps they ought to have a page on their website to advise people of outages - but then I note that their web site is flaky as well this morning, so perhaps people are giving it a hammering trying to find out what the hell is going on.
Just spoken to the people at Vodafone customer support again. They were still being told it was a software problem and were advising customers of that.
I suggested the guy read El Reg, and whilst he was reading the article, he got an email from his people advising that the problem was due to a break in.
A company telling their own people one thing, and the media something else. You really couldn't make it up....
I'm in Westminster and also affected. I've got full data capabilities on my phone but no voice/text at all.
The ever so helpful callcentre explained they are doing some essential maintenance, yeah right.
PS. To the person pointing out a single point of failure...That is why some organisations pay for dual bearer services ;-) If you need that, then you pay for it
Tried to send some texts last night at about 1am, which failed to go anywhere, Rebooted phone this morning and full signal changed into no signal, so it looks like the fault is reaching quite far west, and i'm near to the devon/cornwall border, and there's not an awful lot west of here.
Good morning to you all,
You may well be aware by now, either from your users, through myself or through the media, that there is currently a major outage on the UK Vodafone network.
Sadly this outage has been caused by the behaviour of a small group of criminals who last night attempted to break into the Vodafone data Centre based in Basingstoke. These individuals failed to breach the building but did manage to cause damage to the site infrastructure, leading to today's issues. That damage has resulted in a loss of service on 5 exchange sites, in turn causing disruption for users from Oxford down to Bristol and across as far as West London.
I would like to assure you of two facts at this time. Firstly, please be assured that there is absolutely no security risk to your business data due to the actions of the afore mentioned individuals. Secondly, the outage is a P1 concern for everyone at Vodafone, from our CEO through to myself and the customer services teams. A P1 category fix is the highest priority and we already have 7 engineers on site with all the hardware they need to execute the corrections. Everyone in Vodafone is on standby to support both you as our customer and to support the network teams in the efforts.
Some users are already restored to full service but others are experiencing anything from a full loss of service to restricted service on voice, SMS, Data sessions or voicemail. It remains our commitment to restore service to all as quickly as possible.
I am in receipt of regular updates from our support teams and I will share this with you as they come in. In the meantime if you could support me in managing your end users with updates I would be very grateful and as always, if you should have any concerns or questions, please do call me (if you can) or E-mail me and I will respond as quickly as I can.
My thanks to you for your patience and my apologies for the issues experienced.
Vodafone just banned me from their eForum for posting the internal information I posted above. They then had the cheek to email me saying:
"Hi,
We’ve removed five of your posts from our eForum today relating to the network outage.
The reason the posts were removed is because the information you’ve received is speculative and has not been confirmed. We are investigating this now and will provide more information as soon as it’s available.
It would be good if you could let us know who you received this message from although we understand if you choose not to.
Please do not re-post this information.
Thanks
eForum Team"
Event happened at 00:40, as confirmed in internal email - yet still no ETA, almost 12 hours later.
http://pastebin.com/GGRi9bHA
Very poor customer comms by VF.
No network status or link on their main page.
Takes about 4 links or a google to get to their network faults forum which was on posted to with this issue @ 943 am this morning.
Customer care lines engaged with no recorded message detailing the problem.
Since this issue started around 2-3a its a very very poor show. (my wife was feeding the baby and saw it go offline).
I used run some medium critical systems for a bank and I would have been shot if it had taken 7+ hours to get the first customer comms out.
Dont these guys understand basic incident management?
Was it a few hundred, or half the country? There's a world of difference between a single cell transmitter going down and an etire data centre that appears to serve most of the (heavily populated) South of England. Hint: One is newsworthy, the other is not.
From the memo, it sounds like the crims damage was to some external device, rather than inside the DC itself.
Given the outage, one would suspect it's the electricity supply thats been damaged in some way, although I would have expected an automated fail-over to an internal diesel generator if that was the case.
If not that, could be the microwave antenna(s) that have been taken out, which could explain the multi-site issue and the difficulty and lack of any fail-over.
I think you're right. Doesn't sound like the servers in the datacentre were attacked.
On a single site basis they're likely to have dual communications, probably both fixed fibre connections AND microwave to the exchanges in question.
However, if the miscreants failed to break in, it's possible that they've damaged the aircon (high temperatures in the comms room would automatically shutdown the servers inside), they may have triggered the fire suppression system, again, this would have shutdown many of the servers.
Power supply would certainly have a backup, but damage directly to the backup kit can take down both the backup and primary, though that's less likely given it will be just as secure as the servers themselves.
Damage to the aircon systems would naturally take some time to resolve. Bringing servers back-up should be quick, but I've seen instances were certain components don't come back cleanly after the type of shutdown that comes from an instant power failure.
All conjecture.
What I still can't understand is that, if this is an MSC, why the exchanges concerned can't be supplied by an alternate MSC. More importantly, why wouldn't this switch be automatic?
This is also hitting their MVNO customers, our contact centres are getting hammered from pretty much all our mobile customers.
What's a little concerning is we've just had an email saying their MVNO incident mgt desk is now knackered and we've been given a mobile number as an altwrnative..
With RIM. I have a number of directors in Switzerland at present. Those with WinMo/Nokia phones are having no issues at all. Those with Crackberries are having intermittent problems sending mail and accessing t'internet and have been since Saturday morning.
Vodafone told me on Sunday morning that this was due to outage on RIM's side and that this is worldwide.
Any one wish to confirm/deny this?
All the people knocking others for having a single supplier of phones get back in the real world not the world your living in.
I work for a Water utility and we have well over a thousand mobiles in use, plus laptops etc using mobile data for work allocation and emails etc.
On top of that we have hundreds of remote sites that have GSM for comunications.
These are done via Vodaphone main centre not local towers at this end
None of these is dual sim etc so doubling up means massive cost implications for remote telemetry plus all the extra sims for mobiles and laptops etc, thats not countig replacing sims.
I'm sure you would be happy to have your water bill raised to cover this cost for something that has only happened once in the 5 years we have used this system.
Come up with a lox/cost free option and fine, if not then just get back to the real world before you post such rubbish about spreading the risk.
After all do you all have Apple Macs on your desks just in case a windows virus infects everything and requires a few hours to fix.....
Its not a question of knocking people who have decided to use a single supplier - this started with someone who had made that choice then started complaining when that supplier let them down. As you say, using one supplier can be a perfectly rational decision arrived at by balancing convenience and cost savings vs risk of failure and costs of that failure. Having made the decision, however, one is expected to live with it (unless of course one is a politician or bank director).
Get some FREE PAYG SIMs from another operator (IIRC Orange and O2 do them for free). Issue these to your staff. If and when your main comms supplier fails, top these up with credit and ask your staff to swap the SIMs in their phones.
If necessary, you may have to get your provider to network-unlock the handsets. They will usually do this for a small fee, or for free depending on the phase of the moon or some such similiar thing. Even if you end up having to pay this, it is a small amount compared to the monthly contract cost.
I have a spare old handset at home with a PAYG SIM in it for exactly this purpose. I would suggest that your only difficulty is in the lack of willingness to consider your DR requirements in the first place, i.e. laziness.
And how are we suposed to do that for 7-800 remote sites on the tops of hills etc ?
Plus what about all those who forgoot where the sim is of lost it in the last 5 years when not needed......
Its easy to come up with solutions when just one office and a few people but not when spread over sveral counties involving more than a thousand phones plus several hundred laptops etc for data.
Plus PAYG sims don't do data much do they ?
Vodafone and O2 do not share a network. They have an agreement to share sites, Air conditioners and power supplies. Each network puts their own equipment into each site, independent of the others, and connect to their own internal network.
There is no overlap - you'll need to find something else to blame I'm afraid.
Just been reading an FAQ about the outage on their forum and saw this:
Q. Have you experienced anything like this in the past?
A. Nothing of this nature.
BS - I remember on the 3rd of August a similar thing happened, a break in at one of their DC's flattened their entire Pager network for the whole day.
http://www.theregister.co.uk/2005/08/03/vodafone_pager/
Vodafone UK sold their pager network a few years ago. Regardless, the pager network is a very different system.
Interestingly, many of the core network management folk do use the pager network simply because if the network is down, mobile phones aren't the best for informing you.
There's also a historic problem that mobile signal was far from 100% even at the best of times so it's possible the engineer lived out of range. That's less likely these days, but pagers are still used in conjunction with automatic alert systems.
Every major corporate data centre I've walked into in recent years has been equipped with 24 hour security guards, blanket CCTV, and alarms.
There are normally several cordons of physical security... including alarmed security doors with keypads or RFID tags, and even steel cages around the most sensitive/critical infrastructure.
In short, penetrating into such a facility unnoticed, and causing serious damage, is almost impossible for a casual sneak thief to achieve.
So how on earth could someone break into a Vodafone data centre, apparently unobserved, and cause extensive damage?
The explanations being given by Vodafone at present don't sound very plausible to me.
The site has redundant capabilities with as correctly stated fibre and microwave transmitters, the issue comes from its almost impossible to expect a entire DC to fall over, if telehouse north went down the ENTIRE UK would lose internet, its not possible to predict this kind of incident, the datacentre in question is guarded by security doors and is in a secret location known by some locals but with one route in and one route out its not a manned site.
The breakin resulted in two outcomes depending on which information is believed a HLR was stolen or nothing was stolen as they failed to break in.
Depending on the methods used to get in numerous protection systems would have been activated some of these will include data destruction to protect CUSTOMERS data - as with most major companies in the event of a breakin data is logically scrambled and or physically destroyed the restoration of this data will take time, if the fire suppression system went of they will have to wait for the chemicals to dissipate (being a unmanned site they will use extreme protection systems which literally remove oxygen in a matter of seconds using very powerful gasses)
chill its not possible to predict this all the backups in the world wont help if a DC falls over unless you expect vodafone to spend some of the 12bn profit on building a redundant site for every DC they own (at a cost of at least 3bn per site!?)
get a grip, if you are failing then YOU need failover, i have a secondary mobile and redirection to a SIP number which i can access via my phone that cost me less than £10 to set up a year ago.
Just wait it will come back up.
Is that the same secret location revealed by Googling Vodafone Basingstoke?
That leads me to a planning application on the basingstoke.gov.uk site with an address and details of the site?
Address details that you can plug into Google Streetview, to discover it is (in effect) a small shed on the corner of an industrial estate?
Rather than cc:ed to all and sundry.
I sent an email of remonstration to the Vodafone contact on the Info Email.
Somewhat (un) amusingly I got the following response.
"I'm currently out of the office with no access to emails or phone.
For any other queries while I'm out of the business, please contact the Vodafone Managed Services helpdesk on 08700 777 533. Otherwise, I will contact you upon my return."
Hey like the rest of us on Vodafone then!!!!
The really weird thing is that while all around me, my fellow Vodafoners have been complaining about no service, I've been getting much better signal all day. I'm on Vodafone and here in Oxford, get virtually no signal in my basement office and rarely get any 3G when outdoors in the heart of the city. Today though, I've been getting 2-3 bars in the office and a 3G signal when out in town at lunchtime. Haven't tried making any calls to be fair, but have definitely been receiving data.
You can have physical security with fences, gates, CCTV, alarms, Robotic guard dogs etc. but there is always the human element.
Bank robbers don't patiently queue up and politely ask the teller if they wouldn't mind filling the bag they brought with cash. They carry guns and kidnap the managers family.
On-site guards are vulnerable in exactly the same way.
While there's no information that this was the case, it could the criminals simply threatened the guard into letting them in. If a villain in a balaclava says "Open up or I'll bash you head in with my baseball bat", who here would say No"?
'@Sid_Emot: Luckily for you, roaming and non-roaming UK corporate customers will be managed from Ratingen in Germany from later this year, hopefully german sicherheits will be better - not happening quite soon enough though.'
That's news to me, perhaps I'll get my redundancy letter real soon considering corp and public sector are managed out of sunny Wiltshire.