Fasthosts hit by severe floods

Fasthosts, which last week suffered a major system outage, saw its woes continue today, with its headquarters hit by the floods in Gloucester. Several Fasthosts customers contacted El Reg alerting us to the downtime, with the service, for some, remaining unavailable for the entire morning. One customer, who has since seen …

[Anonymous Coward]

Other ISP's down and out too eg: Namesco

Fasthosts are not the only ISP's having flood issues, Namesco are too - see:

http://www.ndousers.com/viewtopic.php?id=2450

Please be advised that if you wish to call us, that due to the severe flooding in Worcester, we currently have no power to our Head Office which is our Administrative and Call Centre operation.

We would urge you to contact us via the support ticketing section in your online control panel as we have made arrangements for enquiries to be dealt with.

Please only call if you have an urgent enquiry that cannot wait as we are operating our telephone lines with a skeleton staff compliment at present.

All telephone numbers are now only open between 9am and 5pm Monday to Friday until further notice.

Thank you in advance for your kind cooperation.

[Seanie Ryan]

I pity them

I have over 20 servers there. All were down at 7am. Rang the support line and was number 55 in the queue and got through over an hour later.

Engineer told me that they were working on it and staff couldnt even get to work.

About half the servers came back online about 9.30 and the rest about 1pm

The RSS feed says that phone services in the area are out due to weather.

Fair play to them, cant be easy.

seanie

[Anonymous Coward]

working

Dunno, my fasthosts accounts seem to work.

more power to 'em for it too! (pardon the pun)

[Anonymous Coward]

They're not flooded, they're out of power

We have a site hosted there, and suffered downtime this morning. Our webfirm however quickly moved to an alternate host and we only suffered 90 minutes downtime of our site (though I dont know if our original fasthosts hosting is still down or not).

Apparently the reason was not flooding of the data centre itself, but the loss of power caused by flooding in the area. They had been on generator power but run out of petrol - and the flooding made resupply impossible in time.

It begs the question of whether they should have notified their clients the moment they were forced to switch to generator power to give them forewarning of potential downtime...

[Anonymous Coward]

Fasthost down!

I'm surprised that some bishop hasn't claimed this as God's wrath for all the spam that comes from them!

[Hayden Clark]

Somebody check the spam logs next week

... I'd be interested to see a graph of spam volume over the last week, and into next week, to see if there has been a visible dip.

I've got £5 on 10%, any more offers?

[Anonymous Coward]

Still Down

Still down!

after a call this morning and waiting in que from position 53 i was assured power had been restored and all the dedicated servers were coming back online. 6 hours later still no response and now fasthosts telephone has been left of the hook. Not very good, not good at all! Hello Rackspace!!

[Antony King]

Still down

Some of our FH servers are still out of action - Seanie Ryan must have got lucky having had all 20 servers come back.

[toby powell-blyth]

They have a full scale backup generator

I've seen fasthost's backup generator in action (worked very well, all the servers were fine when I saw them testing it).

The only trouble is that it's in their carpark, at ground level.

With hindsight, this would be a design flaw, seeing as they are nigh on 100 yards away from the dock edge.

[Joe]

Is my server under water??

My server has been down since 7:48am this morning.

No sign of it coming back up :(

[Anonymous Coward]

Other Suppliers

anyone know of any other dedicated server providers, where I dont have to go thru a sales call and a quote proccess. i need a win2003 server with remote desktop access and fast!!.

I'm very disapointed with Fasthosts..

[Anonymous Coward]

Fasthosts service updates - or not

Received an e-mail telling customers not to e-mail but to look at service updates. The latest update is 1pm and it is now 4.45pm and still no server!!

They would get less e-mails if they posted more updates. Interestingly they also asked customers to "bare with them". No wonder they are not getting it fixed.

[Antony King]

Reset ??

"A very small number of dedicated server customers may be experiencing a reduced service due to a system re-set."

What's a system reset them ? marketing speak for 'we've buggered it up and we don't know how to fix it ?'

Some of my servers are still down as of 17:26. Unacceptable.

[Anonymous Coward]

Interesting Wording

Fasthosts have an interesting approach to the wording of their updates.

"A very small number of dedicated server customers may be experiencing a reduced service due to a system re-set. "

Reduced service is obviously a euphemism for no service whatsoever. I've had two servers down since 8am this morning and no idea when they'll be back.

And from their service status page...

"13:00

A small number of dedicated servers are still suffering connectivity problems but we hope to have service restored to these machines shortly."

You've got to love their use of the words "hope" and "shortly". Especially given that nearly 5 hours later this small number of servers are still dead in the water (if you'll pardon the pun).

The net (oops) result is that I for one will be moving sites from my remaining dedicated servers to a new host. If only to ensure that my dedicated server hosting is provided by a business with a significantly reduced exposure to the threat of natural disaster.

At the same time, my sympathies go out to those affected by the flooding.

[Anonymous Coward]

OMG, Still Down!!

Totaly shocking, how can half the servers beup and the other half down? I can see my servers are live in the control panel, I simply can't access them with either Remote Desktop or the Control Panel java applet.

!!!!!!

[Karl Blackburn]

Its 6.30 and still no server.

I run 2 major eCommerce companies and so far todays downtime has cost me in excess of £10k... Please bring on global warming.

[Joe]

Cashing in on the floods...

Not only has my dedicated server been unavailable all day, the cheeky gits at fasthosts have redirected ALL of my websites to a holding page with pay-per-click adverts on!!! so they are making money from my crippled websites while im loosing money... thanks alot Fast (to be replaced) Hosts!!

[Anonymous Coward]

Business As Usual At Fasthosts

The real issue here is not so much that they are having problems, but they way they handle them, i.e. lies and misinformation, if you get anything at all. When this kicked off this morning, it was two hours before they acknowledged a problem on their system status site - in the meantime they're wondering why they can't deal with the 50+ inbound call queue!?

Floods are one thing, and I could forgive the problems, but this is standard behaviour for them, and we've had enough of it this time. We aren't waiting for the servers we still have at Fasthosts to come back up. (Which they haven't as at 20:30)

I guess we are one of the "small number" who "may be" experiencing a "reduced service". That beggars belief.

[Kate Callow]

Some Information would be nice!!!!

Still down and no information or reason given!!! We use Fasthost to host our servers which we have 10+ websites on - they have been off since 6.30am. Tried all day to phone - you get put in a que at number 58! So you try emaiing and it gets bounced so you spend all day watching for updates, and find there is only two for today - what are they playing at they should be keeping their Clients informed of what is happening - what the problem is - is it electrical, is it water flooding - or is it technical????? grrrr I have Clients wanting to know when their sites are going to be live again and have absolutly nothing to tell them which makes my Company look bad!!!

Has anyone got any information on when they are going to be up and running again?

[Anonymous Coward]

At least 5 hours without service

"A very small number of dedicated server customers may be experiencing a reduced service due to a system re-set."

A "system re-set" suggests maybe the odd switch or rack went down. Like the majority of other readers on here, I was without service for 5+ hours; a devastating loss of service. Very sorry to readers still having problems, if your day has been anything like my morning you've every sympathy.

[Anonymous Coward]

Still off!

Like Kate above, I am still awaiting an update from Fasthosts.

Its now 10PM and my servers have also been off since around 6AM this morning.

This is completely unacceptable. While I sympathise with the situation, the lack of information from the company is extremely frustrating.

I have many e-commerce websites hosted on my servers that are losing a lot of money. This makes me look bad in front of my clients and the annoying thing is I cant actually tell them when the server will be back cos Fasthosts havent even bothered to tell me.

This is VERY BAD BUSINESS Fasthosts. I have been really pleased with your service till now but this is just not good enough. Fortunately, I havent yet moved my bigger e-commerce sites (that do £10K per day) otherwise I would have really been in trouble.

Lets have an update please!

(PS. Best wishes to those families effected by the flooding. If you get a spare 10 minutes, can you go and give Fasthosts a kick up the arse for me!)

[john]

My Latest attempts to contact Fasthosts

I have a dedicated server that is still down. I'm really starting to panic now.

I have tried their alternative support number shown on their status page. I got redirected to an outsourced call centre (in the United States) where my call was answered immediately. However, they were completely unable to help as they knew nothing more than was already on the status page.

All they could do was to advise I email their support with my quesries. I have done this but I have heard nothing. I will post here again if I hear anything.

This is going to seriously damage our business, but nothing like the disruption caused to the poor folk that live there.

However, the lack of information available makes me worry of a cover up.

[Jamie Jones]

USA hosting

I host all my servers in America - the ping time from home is only 100ms - I'd love to host them in the UK, but the costs are just too much!

I don't understand how UK hosting companies have any business whatsoever!

[Anonymous Coward]

Your expected to feel sorry for them..

In the couple of email exchanges I had with them today, they came across as wanting you to feel sorry for them, as if it was somehow my fault that they have a data center next to a river.

Email requests have been ignored since 5pm... I'll be watching the news tonight in case there is a feature on a few operators being airlifted from a support center.

Still have two of my server down.

[Anonymous Coward]

have a great day... YEP

Servers down since early morning hours.

Finally got through to somebody at Fasthosts (22:00). Been told that some of the dedicated servers are back up (as I haven't been reading their support alerts, every RARE word of it) and that they can not give an ETA for the rest that are still down, reason being, that they haven't experienced that particular problem before. What ever happened to 2nd/3rd line support from technology vendor? Are Fasthosts so low down the ladder that they can't get the right support?

I really feel for the people in the flood affected areas. and hope that their lives return to normal quickly.

As to Fasthosts, negative experience with support quality is becoming the de-facto standard. To top it all, the chap on the phone finished the call with. HAVE A GREAT DAY. A great day indeed..... In the meantime, I continue to look like an idiot to my clients....

[Anonymous Coward]

23.25 - Still offline

Having been told 12hrs ago, after a 2 hour queue, that my system would be back up "shortly" I am dismayed to see an extra 10hrs have passed with no further info to explain why. Support staff now available again at their call centre offer no information as to the nature of the problem nor when they think normal service may resume. Status updates on the website are woeful and emails are going unanswered. While I appreciate their situation at present, the absence of information is inexplicable - we just want to know what's going on and have a realistic idea of when it may be resolved...I don't think this is unreasonable.

[Anonymous Coward]

Still Offline!!

I can't wait till tomorrow morning, same excuses as to why the servers are still down.

They seem to have shifted the support to USA, the company they have chosen has terrible support.

Grrrr

[Matt]

0000hrs still no server

been down since 6am today. I'm wondering if i can use this as an excuse to get out of a 12 month contract on a dedicated server i have.

Paid for a ton of advertising also which is now down the drain :s

[Anonymous Coward]

Satellite view of data centre location

Out of curiosity I decided to see how close Fast Hosts was located to the docks. The following is a link to their contact address on http://maps.live.com - post code: GL1 2EX

http://maps.live.com/?v=2&sp=Point.sn968qgtc53n_Fasthosts%20Contact%20Address_I%20wonder%20if%20this%20is%20the%20data%20centre%20in%20question%3F__&encType=1

Paste the URL into your browser and then click on the Aerial view. Is this where our servers are located? Note to self, "must check out the height above sea level when selecting future data centre locations".

It would be good to see some current aerial photos or ground shots of how close the water is to the data centre and how they are managing to keep it at bay. Lets hope they have raised floors, underfloor water wells and pumps.

Networking outages last year, power outages last week and now a threat of flooding.. it is almost amusing.. for some reason images of Faulty Towers and Monty Python come to mind..

Best wishes to everyone who is affected by the floods and hopefully everything will return to normal soon.

[john]

Fasthosts Uncontactable - server still down

Since my last post I've not even had a confirmation that they have recieved my email. I have called their alternative support number again, only to get a recoded message boasting how all their systems are fully operational and that there is nothing wrong.

I cannot speak to a person and take issue with this as they have made themselves completely uncontable and are ignoring my emails. Unfortunately I can't do this with my own customers. If service is not running by tommorow morning then my company might well be finished.

Whilst I have every sympathy for those whose properties are flooded, I could loose my entire business, my job, and my house meaning I'm totally homeless.

[Anonymous Coward]

Get a life

Personally I think alot of people here need to get a grip. The whole area is experiencing floods. The worst in living memory. People are then saying this isnt good enough? Get a grip, maybe even a life too. If your site was that important you shouldn't rely on a single datacentre. Also if a customer leaves you over this one issue then he probably wasnt all that happy with you to begin with.

[Anonymous Coward]

RE: Get a life

ahem, do you work for fasthosts?

"Unrivalled connectivity - plugged directly to the backbone of the internet, every server enjoys a fully redundant Tier 1 connection so our customers never experience slow-down or service interruptions. "

is what they say, shouldnt they at least make their cutomers feel "in the loop". it's not difficult to post updates even minor trivial ones, just so that there is at the very least a feeling of progress.

All the best,

[Karl Blackburn]

Its back.... 5.21

After nearly 24 hours my dedicated server has arisen from the ashes.....

I was about to give up hope, read the comments on El Reg... clicked for one more time and its popped up...

[Dr Adrian Steele]

all services allegedly back up and running

Well FINALLY, after nearly 24 hours, our dedicated server is back up and running again. We have lost an awful lot of business inquiries

I need to get this of my chest. The reason we pay for a dedicated server in a server farm is PRECISLELY for this type of event - maintaining continuity nearly 100% of the time. This wasnt needed, or offered, I wouldn't pay £600 a year for it.

With this outage ALONE the uptime has dropped to 99.7%, whereas what we expect is 99.99%, about an hour a year downtime.

Come on Fasthosts, spill the beans, if you didnt get flodded or loose the electricity supply, then what kept the servers down for a day?

[Graham Jones]

24 hours later and they are still off

Fasthosts latest comedy message is that all aspects of their systems are running normally, that would be OK except for the fact that we still have servers that are down.

The support line is on a recorded message as it doesnt open until 8am.

It is amazing though that they dont have any problems with their billing system as they have taken money overnight from my credit card.

Last update to their status page was 14 hours ago, that says a lot!

[Anonymous Coward]

What now...

Currently 3 of my 4 servers are online.

In the past I have not found fasthosts to be that bad for the money I am paying. All companies can have router failures and lines going down; but now it's a little different in that not every hosting company is located in a part of the country that can be flooded a number of times each year and when the flood occurs support is non-existent.

Maybe this is a once every 10 years occurance, but I would not buy an house in a location that is likely to flood, so why should I put my business there.

Be interesting to see how good the fasthosts SLA is. Anyone know what I can claim?

2 servers down for 6 hours.

1 server down for 24 hours

1 server still down.

[Anonymous Coward]

web host choice

It makes one realise that there is more to picking a web host than their website and sales pitch "we are the best, and our triple redundant servers also put the icing on cakes"

- in future, we will all use insurance quote style questions, and ask whether our hosts server location is prone to flooding, and whether their hardware, including backup power are high above the tide mark.

Hayden. £5 on 10% reduction in spam? 5-10% agreed.

Perhaps the marketing companies will move their domains so that their profit tide is not thwarted?

Anyone can move their domain TAG to a new hosting company via Nominet for £15 - for an urgent move to new hosting.

[Ian Browne]

pulling the wool....

I tend to agree with some of the comments above. Of course our sympathies have to go to the individuals affected by the weather conditions but it is the mis-information that is frustrating. We will be moving our services away from Fasthosts not because of the downtime, but because of their lack of information about the interruption of service.

Again this morning it says that all dedicated servers are up and running normally but we are still unable to access our servers. I wish they were more honest about the situation and then maybe they wouldn't face such a backlash

[frank]

Not ideal

So, errr, yeah. Um...

Starting to get a little annoyed now because I can't get any description of the problem, an ETA for getting my server back online and

So far, the 100 or so people who rely on that box for email are being understanding, but I can sense patience wearing thing. It seems to me that over 24 hours of downtime to a server without telling me what to expect or even what's wrong in any detail is more than a little poor.

We are reviewing our hosting arrangements. Somewhere a bit higher-up perhaps.

[Matt Tringel]

Location

We dont have dedicated servers with them but we do have many dozens of acounts with fasthosts and after seeing the mass problems they seem to be having i will now certainly be transferring the important ones!

I appreciate you can't be held responsible for weather conditions but keeping people informed and not putting a datacentre next to a river would certainly be a good start. 1&1 or imagiserv here we come

Cheers - Matt

[Anonymous Coward]

"I run 2 major ecommerce companies"

"I run 2 major eCommerce companies"

Super. So you'll have some expertise in risk management and the use of resilience in various ways to minimise the risk of lost business then.

"and so far todays downtime has cost me in excess of £10k"

Oh, apparently you don't have a clue. Hostsite down = business down?

How much would a bit of advance planning have cost you? A lot less than the £10K you've lost, I'd wager. And that's just the loss so far; if you've lost customers because of it, will they ever come back?

Laughable. Unlike the situation as a whole; my sympathies go to those affected.

[Andrew Leaver]

Terrible Service

I host my company website through Fasthosts and have had no web-presence or e-mail for over 24 hours. We are not a technical company and are now getting calls from clients worried that we have financial trouble as we have vanished from the web. I can sympathise with the conditions in Gloucester but the fact that there is NO information NO telephone support NONE of my e-mails have been answered an yet they post an message on their phone saying there are NO PROBLEMS! Bad customer service is one thing..............lying to your customers is quite another. In the meanwhile has anyone any ideas what I can do!!!

[Seanie Ryan]

total idiots

anybody who complains here about this ruining their business... cop on! you are only making fools of yourselves. I run an ISP and even this didnt ruin my business, so if you are in a different sector how can it have that effect on you?

And if you are large enough that being offline DOES affect you that much, if you have relied on only one host/server then you deserve to be fired.

To all the others who are going to move.. who is to say that the datacenter you move to doesnt have their 'black' day next week?

Would your brains not kick in now and make you decide to not 'move' but to get _another_ server elsewhere and have redundancy. If you cant afford it, then the website ISNT so crucial to your business and your whining is just plain lies.

eg : 10k loss in 1 day and it never crossed his mind to spend 1K extra per year and put in a server in the US?? Come ON!!!

I have servers in Ireland, Uk and USA so no one point of failure. Yes, its an inconvenience when servers are offline, and some clients are lost, but thats bsuniess, not matter what sector you are in, things go wrong and its up to you to decide if the cost of safe-guarding is worth the potential loss.

Base point is : things go wrong, no matter where you host, if you dont plan for that, it your own fault.

I know loads of people wont like hearing what i have just said and i am open to major backlash now, but, to me it just seems like common sense.

[Jamie Jones]

Redundancy!

John, and others, I really pity your situation, and certainly aren't mocking you, but really, I host sites that are FAR LESS critical to me than yours appear to be, yet I use 2 servers at different ISP's and different locations (bombs/terror-threats/ISP going bust etc.) and also have a third cheaper 'jail account' on a third server at a third ISP that I can use to route mail, and put up 'holding pages' if I lose the 2 main servers.

Really, people, if these things are so important, you need to have your own redundancy - don't depend on the ISP!

I do agree about the misinformation, though, if that is the case.

If anyone there has control over their dns entry, if they want, I can host some sort of holding page for them if their problems still exist.. contact jamie at bishopston dot net

[Anonymous Coward]

Up at last!!

Dedicated host has just returned to life @ ~12:40 24/07... Nightmare!

[Jamie Jones]

Re: total idiots

Probably NOT what these people want to hear right now Seanie, but, you're right on the money.

Couldn't have put it better myself.

[Anonymous Coward]

Title

Sean, you are right in a sense, but. I run a smallish company that has grown very fast over the last 15 months. Realising I needed help I forked our for a reputable local company to come in and advise me on my setup, principally on the IT/E-commerce side. Taking their advice I use an ISP for the daily running ie, emails and basic webspace, but the shopping side/pages are all held on a server at fasthosts, who i was assured "are one of the best in the business" and with some kind of backup system also at fasthosts (in another rack???) that is also down.

Now that e-commerce side is currently taking about £500 per day. Small fry to many of you, but still a reasonable sum over the year.

It' basically been down for 2 days now. I've got a page explaining the situation at the front now, thanks to my ISP asking for people to call us by phone/ emergency e-mail. We have 1 phone, a fax line and one mobile. Yesterday I took £224. Today so far £87. One person asked ME security questions to make sure the site hadn't been hijacked!

Yes it's fine to say do this and do that. a) it costs money, b) some of us "amatuers" need help so we pay for it and still get nowhere. c) at what stage do you think, right "IF" my company grows more I need other provisions, and if so when do I invest in that. I work about 70 hours per week. Drawing little wages because I re-invest and have staff and overheads. I paid a Professional, and he also obviously didn't think I needed to go that far yet.

"All our systems are secure and reliable - your data is safe with us" "All our sysytems are functioning normally". Makes me sick. Absolute B*llsh*t.

[Seanie Ryan]

sympathy

OK, i realise my comments were a bit harsh and now is not the time to rub people in it.

I think its great that you build up your business and its growing. I was there once , (still am in many ways) and know what its like to get crap wages while you keep putting money back in.. it seems endless.

But, losing a day here and there for some reason is ALL PART OF BEING IN BUSINESS. Some Newbies just dont get that.

Tomorrow , your own Elec supply could be down, do you have your own generator? The next week you could be broken into and have all important equipment stolen, after that it could be a fire. All these things happen and affect your business.

Reality is that if you run your own business it YOUR decision what redundancy to put in, not someone you hire. If you cant afford it right now, then fine, but be prepared that if it goes wrong you might lose a day and get on with it. Now you might be able to say, well, at £500 a day, its worth putting in a server in the US for $100 a month.

i personally think fasthosts have done well considering the circumstances. Yes , maybe some better information flowing, but come on, put yourself in the same position.. it aint easy.

[Neil Wilson]

Continuing difficulties

Is anyone else out there still having problems with the Fasthosts shower?

I agree with most of what has been said but I do have to take issue with the sanctimonious Seanie - after having congratulated him/her for having been so clever, of course. He should understand that not all problems can be solved by having a server on the moon. Our server has been up and running all through this problem - the issue is with the DNS. Type the web address in and you get nothing. Type in the IP address and you are straight to the site. Now, how would a server in Alabama have helped us with that I wonder.

The last I heard from Fasthosts, this problem is my fault because I must have 'done something with the DNS settings'. Genius.

[Matt Horrocks]

DNS

Well, having redundant servers and running DNS yourself rather than letting fasthosts do it would solve that issue...

[Anonymous Coward]

Fasthosts - Hand of God - Judgement

After what Fasthosts did to my company's web site in 2005, I feel that there is divine intervention going on here.

They shut down my .com business for seven days without explanation except for automated emails from it's 'Abuse' department. (They just happened to be sent to the email address they had shut down.) The automated email said that any attempt to contact the company in any other way would lead to expulsion, I think they called it 'termination.'

When they found it was simply a question of spam in the Inbox (not from us) they did not apologise, they lost the backed up emails and tried to charge me for the non-existent technical support. Dreadful company - bad management, mean spirited, slave-machine.

Hope you're not waving.