Hospital radio station struggles with Yahoo! email 'blockade'

A small UK-based hospital radio station has fallen foul of the overzealous spam-blocking policies of internet giant Yahoo!. Reg reader Brian Purvis looks after the IT needs of Radio Haslar, a hospital radio station that (as so many in the UK) operates as a charity. Over the last few weeks email messages sent to either …

[Anigel]

Smacks of a con

When you have to signup for a yahoo account before they will let you fill in the form they make you fill in to stop them from blocking your legitimate mailings, it smacks of a con to get more people to signup with them.

I run an email server for an online game that has been about years and has never been used for spam and we regularly see these messages in the log files.

Admittedly they normally only delay us by minutes until the next delivery attempt but as for them getting delayed by upto 24 hours, I have never seen that on our systems.

[Neil Hoskins]

Who's their provider?

<yawn> Here we go again. So what if it is a "private, closed relay"? They still have a provider, don't they? They're still paying somebody, unless the Internet has suddenly become free. What if (hypothetically), their provider is spammer-friendly? If I'm a taxi driver and you're a perfectly respectable citizen but you happen to live in a crime-infested slum neighbourhood and your landlord is a crack dealer... am I obliged to take you home? (disclaimer: this is entirely hypothetical as I have no idea of the reputation of their actual provider)

[Anonymous Coward]

"Zero tolerance on incoming spam"?

How do you do that? Inquiring minds want to know.

[this]

This is happening to us too

Yahoo are similarly blocking emails from us to a relative who has the misfortune to have signed up with BT for her broadband connection and was thus sold into slavery by BT to Yahoo! as an email provider at some point, without her knowledge or consent.

We don't know why, but all attempts to forward emails from her website domain to her 'locked in' BTopenworld/BTinternet/Yahoo! mailbox simply vanish - blackholed - with no indication of what happened, no bounces etc.

Actually, we are still unsure of her actual legal email provider as her address is still @btinternet.com but trawling through the vile Yahoo! webmail site seems to point the finger at them.

It took several weeks to figure out why her emails were silently evaporating and since then Yahoo! have plainly failed to read any of our emails pointing out their mistake and instead present us with an automated process of form filling and various other hoops to jump through by which we must prove our innocence (of spamming I presume). They do not have a telephone for obvious reasons.

God kows what other email is vanishing - we don't - as they are obviously chucking it away like those poor postmen who have nervous breakdowns and flush the mail they are supposed to deliver down the toilet.

I could go on, but basically Yahoo! mail is rubbish and their administration is as incompetent as it is Kafkesque - no wonder they are in bed with BT - they are soulmates.

Needless to say we have the MAC number and are moving our relative away from both BT and Yahoo! asap. There's another weekend gone...

[Anonymous Coward]

Resend?

So Yahoo! (or their blacklist suppliers, since it's not just them that Haslar is having trouble with) have decided that this place originates too many mails of the same content and has labelled it spam. So they ask them to put yet more repeat ("looks like spam") mail on the wires?

Riiiighhhtt.

[Damien Jorgensen]

one person

all it requires is just one person on the mailing list to have closed their yahoo account, send a few messages to that person and you're banned

[Robert Laverick]

Not just one site hit

I've been having this problem too with a few sites it seems that looking around the results google can find there are a number of sites that have this problem.

Gotta love baddly thought out spam protection.

[The Cube]

This is perfectly normal for Yapoo!

This is not the only domain being blocked by Yahoo! for no good reason. The issue is that too much spam gets through to Yahoo! users and instead of investing in a useful and capable spam assessment regime Yahoo! simply keep lowering the bar for spam assessment, this results, quite inevitably, in a high false positive rate. This is what has happened to this charity, actions such as instantly blacklisting entire domains are much cheaper, in software, admin and processing costs than real spam identification measures.

The real solution is for users of Yapoo! to wake up and realise that they are getting what they pay for, nothing, and that if they want to be able to send and receive email reliably they need to pay for a service. Yahoo! are by no means unusual here, Hotmail and many of the 'free' email services bundled into low price broadband offerings suffer the same issues.

Buying a real email inspection service is in the £1-£2 per user month range, if this is too much to pay then clearly your email is not that important to you and you should stop wasting Internet bandwidth accessing it.

[this]

@The Cube

You're wrong about the 'get what you pay for' jibe.

The person I am trying to help has a relatively expensive BT broadband connection and was hived off to Yahoo! by BT - with no consent. Presumably BT get the 'service' for nothing though. (or get paid, which would be a nice little earner for BT and its captive audience of 4 million tied-in punters. ) That's what I meant by 'sold into slavery' ...geddit?

[Anonymous Coward]

Useless

We've tried in vane to get yahoo's help.

They don't give a flying squirrel.

As far as I can tell yahoo don't want to accept mail - they'd much rather defer it even after we've jumped through all the hoops.

[Nano nano]

Yahoo's London telephone number

I only got some action on a blocked Yahoo account when I phoned the Yahoo London office number.

http://uk.docs.yahoo.com/codeofpractise.html

[A J Stiles]

All you need is

All you need, if you want to be able to send and receive e-mail on nobody's terms but your own, is the following:

* a "wires-only" ADSL service with no ports blocked and at least one static IP address [about £30 per month]

* a domain name hosted with a company who provide a web-based control panel for setting up your zonefile [less than £5 per year]

* a machine with a processor of 500MHz or faster, a hard disk of 4GB or bigger and at least 64MB of RAM [try a skip]

* exim (for SMTP), dovecot (for IMAP and POP3) and spamassassin (for spam filtering)

Looks a lot more professional than a yahoo! e-mail address, too.

[The Cube]

@ By this

Sorry to hear about your issues and the BT victim you are trying to help (and I understand I am also a BT Business Broadband victim and they wouldn't know service if it walked round their offices with an automatic weapon).

Unfortunately, my comment still stands, even if BT are ripping off your friend for an overpriced connection the email is 'free' bundled into the service. There is no incentive for BT to make this useful or effective as the quality of this 'free' service is not a selection factor for customers when choosing a provider, only if there is no service at all. There is, however, a clear incentive for BT to deliver this at the lowest possible cost.

Now you can blame the operators or the punters but in our current market where those ISPs who do offer real service, without crap 'free' services at a reasonable price get hardly any customers whilst the masses line up to be shafted by BT and Sky where is the driver for change?

What would assist is some press attention identifying the basic failures of these 'free' services so that punters realise that they have the same value as their cost (<£0).

P.S. I am about to move house and no, I won't be buying any IP service from the provider who is rhyming slang for 'Out of Order'.

[Nick]

Not! Just! Yahoo!

Its not just Yahoo who do this, we've had similar issues with AwOL and freewannaange. SPF has helped in our instance.

[Dave]

Sounds about right...

I first noticed this when i rebuilt the mailserver at the office and asked a collegue to test it from an external (his Yahoo as it turned out) account - the message queue showed a failed message/deferred, which eventually went through.

We have a static IP, fully properly resolvable DNS on the mailserver (a common spam testing trick is if the email originated from an SMTP server which is on a dynamic IP or if it's HELO is not its actually internet resolvable hostname) and it still does this.

Yahoo seem to operate an 'guilty unless we say-so policy' This is why i operate an independant-from-provider email address/server (both at home and at work).

My home server is setup so that if i change ISP everything will still 'just work' as long as i update the public DNS record (i didn't even have to do this before when i was on a dynamic IP)

One solution you could try is use your ISPs outgoing mailserver instead of your own for sending mail - this worked a treat for me, my local SMTP server simply relays all of the allowed outgoing mail through my ISPs server instead of attempting direct delivery, this then just looks like you have sent the mail normally from a client.

[Anonymous Coward]

Yahoo, Hotmail and AOL!

We have the same problem with Yahoo and got a list of questions.

To begin with I had to set up a yahoo account to be able to fill out the contact form. Yahoo person thought I had filled out the form for the Yahoo account I'd just opened, even though it was damn obvious it wasn't.

I'd given them enough information on the first form, there was nothing else I could give them, but I also got the long list of questions.

Found out Hotmail also suffered from a similar problem. You have to notify Hotmail about your SPF record !!! Yeap! Even though you'd think it would be possible to check it just like everyone else!!!! F@$k sake!

Christ - don't even get me started with AOL.

I think the problem is because people are now told never to unsubscribe themselves, because it lets the spammer know they have a live account - people are using the 'report as spam' option FAR TO OFTEN!!

Now if I see an email from a hotmail, yahoo or AOL domain - I don't even waste my time to reply, after all you never know if they will get it or not! Grrrrr!

[Robin Traylor]

Yahoo! No. Yaboo!

I run a small mail system with MDaemon which handles a few domains and have no end of problems delivering mail to Yahoo. It's very often that I'll look at the system to find a number of mails all held in the queue and checking the logs shows exactly the same thing.

My system is very clean and locked down against being hijacked or used by spammers.

Yahoo is far too cautious and blocks far too much genuine mail but they don't care about it as far as I can tell. I've been in touch with their boffin techys and they just tell me to retry sending. Mail does get through eventually but it's like a grey list gone badly wrong.

[Ian Ferguson]

Wow, you got an answer

I've had similar problems trying to un-blacklist legitimate email domains from Yahoo and BT before, but never as much as got a human response from either service provider. Nowadays I've given up and if customers complain they aren't getting emails from us, I have to tell them it's their overzealous providers' fault and there's nothing I can do... in other words, I badmouth them (BT particularly) every single day. That's what lack of customer support gets them :-)

[Anonymous Coward]

Ha...

BT/Yahoo black listed the ISP i work for, and it's a running issue with them...

(we're big, we're probably your providings in one way or another).

We're just as bad though, our mail servers blocked our own domains once when we had a new back end system installed.

[system]

SPF

Could the problem be with the SPF records?

The SPF for radiohaslar is:

v=spf1 a a:mail.adsl.radiohaslar.co.uk a:mail2.adsl.radiohaslar.co.uk mx:dnsmadeeasy.com mx:hostgator.com mx:radiohaslar.co.uk; include:hostgator.com ~all

Basically, all MX servers for hostgator and dnsmadeeasy, as well as any servers included in hostgators SPF are authorised as senders for mail from radiohaslar.co.uk. Anyone with an account with those 2 services can send mail with a fake header claiming to be from radiohaslar, and it will be accepted as the truth.

Enough spam from those servers, and any email provider is likely to take your word for it and treat it as spam authorized and originating from your domain.

[Mike]

reccomended fix

Took a quick look at their setup.

The A record for the MX, the Hostname in the SMTP greeting, and the PTR don't match. get them all matching and the likelyhood of getting mail through will be much higher. Can't guaruntee it will work for yahoo, but some mail servers will definately be blocking their mail on the above points.

[Anonymous Coward]

Same story Here

I'm a network administrator/director for a non-profit Usergroup/ISP in Houston, Tx. (http://www.hal-pc.org)

We've been having the same problem with them off and on for months. The major difference is that with 15K accounts on our mail server, we spool mail for way more yahoo accounts than most people and when they block us, our mail spool increases into the thousands very quickly and slows our servers to a crawl. I'm sorry to say that the 17 item questionairre is the a form response and not the result of the reg article. Yahoo doesn't care about interoperability (domainkeys) they obviously don't care about customer support and they don't care about bad press. They get plenty of bad press, but it's generally justfied because they pull stuff like this.

I've been through 15-20 email transactions with them where there are three constants:

1. I jump Through their hoops

2. They send me form resopnses that are not applicable to my situation and demonstrate than nobody read my responses.

3. If I pay them, they'll let me send whatever I want.

Their customer service number (866-562-7219) is completely pointless unless you work your way into the premium webhosting queue and their main corporate number (408-349-3300) generally results in a transfer to their customer service number if you don't know the name of someone you need to be transferred to.

I have a feeling that they'll do this until a critical mass of ISPs cannot send email to them at which time:

1. They'll die.

2. Having been in bed with SBC (AT&T now), they'll use the inability of small ISPs to send them email as a selling point for AT&T to further drive small ISPs out of business. and then they'll die.

YAAAAAHHHHHOOOOOOooooooOOOOOOOOoooooo!!!!!

[Cheshire Cat]

Looks like Yahoo use Greylisting

This sounds like Yahoo are using greylisting to me, which we also use here at the university.

Basically, all incoming connections that originate from (what appears to be) a home account or an ADSL are initially sent an SMTP '421 Defer', and when the connection is retried 20mins later, it is accepted. Once one email has been confirmed this way, the source IP address is whitelisted so it doesnt need to be greylisted again.

Spammers (and zombified PCs) are sending so many million emails that they can't defer and retry - they treat a 4xx code as a 5xx (fail). This cuts out about 45% of our incoming spam at the SMTP conversation stage (less than 1% of greylisted connections are subsequently confirmed). This is a good system for mail gateways as almost all valid incoming email is coming from another mail gateway, and it blocks almost all zombie-PC originating spam.

This method is not appropriate, though, for ISPs to use on their clients because (of course) their clients are all on this sort of IP! I suspect that Yahoo uses greylisting for its web-based accounts (sensible) but now they have started to manage email for ISPs clients it doesnt work any more. They should explicitly exempt from greylisting the IP ranges of the ISPs for whom they manage the email.

Steve

[Morely Dotes]

Lies! And possible explanations.

"Yahoo person thought" is such an obvious falsehood!

And one problem that should be obvious: "mail.adsl.radiohaslar.co.uk" contains the string ".adsl." which is about 99% certain to indicate it's a source of spam.

Furthermore, hostgator.com is hosted by ThePlanet (aka "ThePlanet of spam" among those of us who've had to deal with their effluvia for years) and I see that radiohaslar.co.uk is associated with hostgator.

[Steve Roper]

Security doesn't prevent spoofing

Part of the problem is the "domain spoofing" spammers use to push their crap. No matter how tightly you secure your mail servers, you cannot stop a spammer from spoofing your domain in their email headers, and then forwarding it out through their botnets.

Case in point: We were suddenly deluged with "Your message has been blocked as spam" autoreplies from addresses we had never heard of or sent anything to. One of the accounts on our server is a "catch-all" account for capturing emails sent to our domain with misspelt email addresses, and it was this account that was hit with all these rejection messages. When we did the analysis, we found that a spammer was using multiple nonexistent addresses@our domain in their headers, and sending them from a different server with spoofed headers. Example: my email address is stever@[ourdomain].com, so some of the block responses were to emails allegedly from "stevebomb@[ourdomain].com, steveblow@[ourdomain].com, stevebar@[ourdomain.com]" - none of which exist, nor were ever set up on our servers, nor did our server logs ever show any sign of penetration or hijacking - ergo, our mail server was not responsible for the attack. Somebody used another mailserver (or a botnet) with a trojan to modify the email headers to generate names exploiting our domain. Result: everyone thought the spam was coming from our domain and blocked us. We had the devil's time and a huge cost in clearing our name and assuring our customers we had nothing to do with the recent spamfest.

Now I am generally against the death penalty, but I sincerely believe that this kind of fraud should result in public executions. These bastards don't give a fuck that they are ruining someone's business reputation, destroying their livelihood, and maybe destroying hundreds of people's livelihoods by costing them jobs; and after the fiasco I had to straighten out, I would have definitely gone to see the public hanging of the bastards responsible and bayed for their blood with the crowd. If I am ever lucky enough to meet anyone responsible for this kind of spam spoofing I will introduce them to my stainless steel bash bat, law or no law.

[Mike]

Greylisting

Let us know how that works out when you have many subscribers on, say, gmail. Greylisting, at least as she was implented a few months ago, works on a per-IP (or sometimes hostname) basis, so when someone (or their company or ISP) uses a bunch of load-sharing servers, the resend has a good chance of coming from a (slightly) different source, so gets deferred again, and again, until by sheer luck it repeats an address/hostname within the accept window.

I've seen it take hours from a gmail account.

It wouldn't be difficult to fix, and perhaps has been by now at some installations where the admins have a clue. Oh, right, that was the _other_ universe.

Mike

[Nick Ryan]

Re: Security doesn't prevent spoofing

What's exceptionally frustrating is that many of these so called anti-spam "solution" and "mail servers" are written by complete muppets.

For example, EVERYONE with more than one braincell and any spam experience whatsoever knows that the "reply-to" / "sender" address are "trust" fields and therefore can be filled with whatever the real sender feels. As a result, blocking by "reply-to" / "sender" is just pointless.

Mail server software developers are also an incompetent, dumb-ass bunch - as you get typically get hundreds of "no such user" or "this is spam" responses but the software writers don't bother to include the full headers of the original message (typically just the "sender" address). As a result, you've no way of knowing whether the damn message did originate from your systems or not.

I manage a corporate network and one of the first changes I put in place was to alter the corporate firewall settings so the one system on the network that can send and receive mail is the mail server. If you don't do this, and you run a corporate network then you're insane. Before I did this (and to a lesser extent cleaned all systems and installed a decent AV solution on every system), our domain was continually being black listed as a source of spam. Since then we've been fine - we've had the usual spoofed "sender" addresses but there's nothing that we can do about that.

[Brian Purvis]

Fair Points

Hi,

I have taken the comments listed here on board and have already taken action to implement the following.

1/ Amended SPF record so that only the primary mail server is authorised to send mail.

2/ Amended the reverse DNS infomation for mail server IP address to a non adsl one.

3/ Amended hostname of mail server so that it does not display adsl

4/ Made sure that that the A Name MX record, SMTP greeting / hostname and reverse DNS for IP all match.

Although the main website is hosted with hostgator.com they do not host our email, and to be honest their service as a host have been second to none, just a shame that their backbone provider has an allegedly obvious dubious record with regards to spam.

Many thanks for all your help and comments.

Regards

Brian

Hospital Radio Haslar

[David Barrett]

Good luck to them...

I have been speaking to yahoo for the last 3 months to get this resolved, Im not sending bulk mail, maybe 20 messages a day to one yahoo user...

So far I have had to explain no less than 3 times that I am not sending bulk email to their network.

As for their spam policy the "Challange" Rule is Ok in theory... Basically if its spam and it bounces the sending server wont try to send it again, a legit sendinmg server will que it and re send... the second mail should get through...

In Theory this is fine... In practice the yahoo mail server forgets that the first mail was deffered after about 5 mins so the sending server needs to retry the delivery almost immediatly after the first bounce...

Of course this still doesnt fix it - but it does increase the chaces of your mail making it from 1 in 10 to 2 in 10!!!

Its a farce.. If I didnt have clients on yahoo I would have nothing to do with them.

[Mike]

Let us know how you get on.

anyone else having a problem post it here any maybe we can take a look and throw ina few pointers?