Royal Mail sites hit by downtime cock-up

Royal Mail’s three main websites have been unavailable since last night, forcing millions of customers to get used to the taste of glue while guessing where their registered post might be. Blighty’s national postal service currently has no online facility. Instead, customers attempting to gain web access to Royal Mail are …

[Simon Greenwood]

Who run the Royal Mail website I wonder

Ah, the favicon on royalmail.com is a bit of a giveaway...

[Craig]

Smartstamp

Their Smartstamp servers were up until around 11:00 then went down to join the rest of Royal Mail. Looks like it wasn't a big-bang failure then but they've taken everything else down, just in case.

[Anonymous Coward]

So...

their web page is in the post then?

[Anonymous Coward]

"I couldn’t possibly speculate without being in possession of all the facts"

That's fine, please get all the facts, and THEN speculate for us.

[Dave Bennett]

Simon...

Simon, who's icon IS that? I have seen it before but can't put a name to it!?

Thanks,

Dave

[Anonymous Coward]

site down time

The Royal Mail site has been downed by a piece of faulty hardware in a date centre in copenhagen, RM's IT provider CSC is ment to be working on the issue.

[Kenny Millar]

It'll be up in 24 hours

If they tell you it'll be up by 9:30 tomorrow morning, you can actually expect it at 14:45 on tuesday.

[jai]

gotta say though

i was dreading using the phone to trace a parcel - i figured i'd be on hold for hours - but it only took 5 minutes of "please press 1 now, otherwise hold the line" before i got to speak to someone

they weren't able to find my parcel either, but that's besides the point - the phone did prove to be a viable fallback device

[FlatSpot]

Fun place to work at RoyalMail?

Like who exactly are;

Philatelic Customer Services and Smilers Customer Services!??

Sound like new age funky departments to be in!

[Vikash Joshi]

FAO Dave

http://www.sun.com/

[Graham Hawkins]

@Kenny Millar

It might be up in 24 hours, but don't forget that you'll have to take the card, and some id, to the nearest sorting office, between the hours of 7:00am and midday, on a normal working day, and pick it up yourself...

[Anonymous Coward]

Bad 'time of the month' for Xansa..

.. or F.I. as they used be called.

[Anonymous Coward]

Funny that

Today is the e-business leaving do

[Anonymous John]

"We apologise for any inconvenience"

I hate that phrase. "Any"?

[Slaine]

untitled - or just no postcode

"A spokesman told us that Royal Mail tech bods are investigating the problem, but was extremely vague when asked to provide further details"... well it's like a sort of problem thingie and it's proving to be problematic to us because, err... well... it's causing a bit of a problem, especially to our, um, problem solvers.

<wonders> did anyone check to see if the cleaners put the plug back in?</wonders>

[KarlTh]

@anonymous coward

What sort of crap excuse is that? Whatever happened to redundancy, clustering, virtualisation? A hardware failure shouldn't bring down an important service in this day and age.

[OpenSauce]

Not been right for 2 weeks

I have sent a dozen items in the last fortnight by Recorded Delivery or Special Delivery. None of the tracking numbers have appeared on their website (but luckily have been delivered).

[DR]

it's back now

yay I can finally find out if my local branch is open tomorrow morning!

[Martin]

@KarlTh - It's not an excuse...

...it's just an explanation. I suspect our AC entirely agrees with you, but he's (or she's) an AC for a good reason.

[b shubin]

Bean count

@ KarlTh

the way it works is, the primary may be big and beefy, but the secondary is less than 50% the spec on everything, starting with memory. the accountants always want a word when that failover box is purchased, as an idle system already gets their thongs in a knot, and a well-specced one is simply inconceivable (credit: Princess Bride). all one can do, as the original system scales up to 90% or better utilization, is:

[1] put forward a funding request for an adequate failover system every year (it is, of course, rejected, or deferred until next year), and/or

[2] bring up a JBOC* (built from spare parts and scavenged components) tertiary that would take some of the load, but is unproven, unsupported, unauthorized, and off the books (management will force IT to ditch it if they find it, and the persons responsible may lose their jobs), and/or

[3] pray (this will not work, but makes everybody feel better, until the primary "falls over and catches on fire").

odds are, CSC are replacing the failed components in the primary, and possibly doing a bare-metal restore (assuming they have a backup). it is also possible that the primary's UPS had a catastrophic failure (power "flapping", or similar), that may have hosed the primary system quite thoroughly, without actually baking anything. in any case, it all sounds exceedingly unpleasant, and the setup was probably planned with a good measure of delusional optimism.

to conclude, this happens, far too often, and is in most cases a management failure, but occasionally a design issue, and very rarely a technical fault (assuming that equipment has a finite lifespan, which all equipment does, component failure is part of the normal scope of events).

*Just a Box Of Components

[Anonymous Coward]

Collect in 48 hours

"Apologies, we were unable to deliver this webpage to you today. As a result the webteam will have stored it overnight in the boot of their car and will return it to the sorting office sometime tommorrow. Please collect it from there after 48 hours ... please bring a form of identification for you for the IP address to which the page was meant to be delivered"

[Bayleaf]

@ Flatspot

FYI, Philatelic services are to do with stamp collecting and Smilers are a particular type of stamp. The first term has been around for years but the second; well, it probably did come fromt the whale song department.

[Nic]

Should have gone to Zeus

http://www.zeus.com/

[Anonymous Coward]

Speculation?

It's probably just a glitch as they test the new profiling system.

[Anonymous Coward]

Redundancy

If this is a failure of a piece of infrastructure equipment, then as far as I'm concerned the muppets get what they deserve. How stupid to have a single point of failure in their infrastructure. But what have we come to expect over the years from Royal Mail..incompetence.

Doesn't look as if they have heard of fault tolerance does it?

And as for secondary systems only being half the spec of the primary..kinda defeats the point doesn't it, one would expect the back-up system to provide the same level of service, which obviously means the same spec equipment!

[KarlTh]

@b shubin

Yeah, but the secondary system should still work, if slowly and prone to timing out. This is completely down; it looks like there is a single point of failure and it's hosed.

[heystoopid]

Blame it On !

Blame it on all that Hot air from the Home Office Wacky Jacky diverting all system resources required for her to implement her version of Police State Web 2.10 !

[b shubin]

More on secondary

@ KarlTh

you underestimate our fine friends from finance. the secondary is specced at less than 50% on EVERYTHING, including disk space (for which they may have bought a system with 1 big hard drive, if the purchasing is done by the purchasing department and not by IT - not a joke and i've seen this happen). the application may not start, and if the disk got full from replication, the box will hang on boot (if it is a Windows box).

also, depending on what OS it's running, the server may simply drop requests under high load (MS Server 2003 did that when running a commercial Java app server, in a test i've seen).

to conclude, if the secondary is substantially inadequate, it may not boot, or may not be usable if it comes up.

[david]

fallback system

"they weren't able to find my parcel either, but that's besides the point - the phone did prove to be a viable fallback device"

well duh - they were checking it on their web based system ...

[Anonymous Coward]

Hmmm...

This reminds me of the famous British Rail (snow on the line) excuse. Perhaps the bosses will claim that they would have been okay, except for the fact that online customers were ordering the wrong sort of stamps.

Mine's the one with the self-adhesive back panel.

[Anonymous Coward]

Maybe...

this is part of their post office closure program...

[John]

Email

Please email me some stamps, I want to write to someone, oh hang on a minute.....

[Anonymous Coward]

Deutsche Post's website

I bet this wouldnt happen on the Deutsche Post's website.

They even have a version of the website in English, not bad for johnny foreigner! (probably a cunning trick to discredit the Her Majestys Finest)

just shows you empires are not built on gritty determination, ruthless efficiency and razor sharp tailoring, but by fate, luck,stiff upper lips and putting on a jolly good show?

[R Callan]

must blame hardware and bean counters

WE cannot blame winders this time. According to Netcraft all of the listed sites are running Solaris 9/10.

[Anonymous Coward]

As good as down for weeks

The speed of response of the Royal Mail site has been attrocious recently.

Three minutes to get the second class postage page, fifteen minutes and still no response from the print-your-own postage log-on pages.

As far as I can see there's nowhere on their site to complain about the website - just late deliveries etc. How glad I was when up popped a "Take our Survey" and I was able to feed back...

Also suprised that the Royal Mail traceroute goes from UK to New York to Holland - what no UK servers

So maybe they've taken it down to add a second processor and a few Mb more memory

[Anonymous Coward]

theres a jolly little song

..it ends "C-S-C, k-e-y, m-o-u-s-e!"

Joking aside, don't be in a rush to tar+feather CSC until you know detail of the SLAs in the RM contract... oh alright, you get the tar, I'll pluck the chickens