BlackBerry services splutter back into action, again BlackBerry users should be getting the message again today – RIM reckons it has found and fixed the latest fault although there's still some backlog to be cleared. The company tells us that yesterday's outage (as opposed to Monday's down-time which remains unexplained) was down to a "core switch failure", and a failed back-up …
Has anyone actually thought RIM have been compromised? Why else would they be silent with no feedback to their customer base, surely if it was a simple hardware malfunction then they would have identified, fixed and provided a response, coupled with the fact i'm sure their DR plans would have kicked into play to save them the embarrassment of a lengthy outage. Drama over! However they need to address their total lack of communication... The silence is deafening and echoes the RSA debacle... They were breached and thousands of customers were left in the dark. RSA stayed quiet whilst it could get their PR teams in place with how best to deflect the heat and spin and also leave it long enough for people to find something new to talk about before releasing their official statement. They knew they'd been hacked the minute it happened and what they needed was time...
IF RIM simply had a server fail it wouldn't take down a whole network, i'm pretty sure the enterprise beast has more resillient DR strategies in place for this not to be effected... Which spurs my thoughts even more....
I'm intrigued to hear what Blackberry are going to do in order to redeem themselves from this one!
"Core switch failure and a failed backup" says RIM...
What they actually mean is that they have failed to incorporate adequate resiliency in their infrastructure, and they have similarly failed to carry out appropriate disaster recovery testing that would have identified such issues.
I'm getting about a 2.5 hour delay at present on emails. I just got an email through on my blackberry that was in my inbox at 8:30 this morning (11:00 now). I sent myself a test mail at 9:00, still not got it on my blackberry yet.
It's not a huge impact for me today as I'm at my desk but it was a right pain on Monday when I was out and about.
Technically they don't have to provide any sort of SLA to there customers, atleast standard consumers. Businesses may be different. You guys need to remember that the Blackberry by its very nature is a business device and has only really become main stream in the past couple of years.
So what they lost there servers for a few days. I can't remember it ever happening before which aint bad considering how long they have been around. Give them a break jeez!
that the switches in question are managed by some MegaMouth Consulting business improvement partnership, who add value to the proposition by putting in PHBs all over the place to provide business continuity assurance within a integrated world-class best-of-breed process governance framework.
What will have happened is that PHB1 will have asked PHB2 to apply a scheduled upgrade, wiht a backout strategy in place.
PHB2 knows a bit about backing things out and in turn asks PHB3 to apply the upgrade and backout if necessary.
PHB3's knowledge of backing out is running for the pub toilet when he finds the gorgeous bird he's been chatting up is actually a transvestite. So it's something he prefers not to talk about in case his wife finds out.
So PHB3 tasks PHB4 with the upgrade, who in turns tells some downtrodden, overworked and untrained PFY to apply the upgrade. Do the core and redundant systems at the same time - delivery achieved quicker, time is money, young man.
So the PFY does his best, it all goes horribly wrong and it's taking days to fix. Guess who'll be fired for the disaster, though?
I bet you if you look over the last few years, RIM has probably massively increased it's marketing budgets (to attract the youfs) and amount it pays to shareholders. Same problems as Nokia. Use Business school grad rubbish like six Sigma and all those other processes instead of concentrating on being a TECH firm.
Business school grads shouldnt be let within 1000 miles of a tech firm. Don't worry about the product, the product doesn't matter, its ALL ABOUT THE BRAND. Thats fine for Nike and other trend products that will be forgotton within a month but for tech firms its unforgivable. Nokia AND RIM have forgotton that their products are critical products FIRST and a brand SECOND.
I don't care which celeb is using which phone, it's a phone first and if it looks nice, even better. The Playbook is an example of what is wrong with RIM. At some point, someone MUST have said "This is a piece of shit, we are an email company and this doesnt do that" and was either shot down by management or marketing gave the technies a date to meet for release and refused to budge it so an incomplete product was released.
That is what is worrying abotu Apple. Steve Jobs was able to shoot down ideas or push products and override marketing and release dates. Will the new boss do that?
How many technies do you know that will go to a Marketing Director and say "The produc isn't ready and won't be ready for yoru release date. Get back to your 2 hour drinkign lunches and we will give you a call when we want you"
It is a common mistake to attribute an outage to nefarious hacking when simple human stupidity is a more probable cause.
RIM somewhere attributed the problem to a software upgrade gone bad. Having seen similar problems first hand in large (carrier) networks, the long outage doesn't surprise me. It ends up being like a self-inflicted DOS attack, and the more you try to fix it the more the problem cascades due to the law of unintended consequences.
But hey, it might be wily hackers after all.
I think the key point here is transparency and communication. Every company that provides a service will at times have problems with providing the service as advertised.
What is particularly bad about this is how poorly RIM are communicating with their customers. All the money being spent on building the brand through advertising is being unravelled by a failure to communicate.
At the very least I would expect them to have at least one person posting updates on their website and twitter feeds.
The two (count them! two!) updates posted so far on their website over the course of a three day outage are laughable for their lack of comprehension of the scale of the outage and the lack of information about what they are doing to remedy the outage.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
