back to article Google says its AI will jetwash all traces of malodorous spam from your box

Google has proclaimed its rightful place among the Gods by creating an artificial intelligence so sophisticated it can block 99.9 per cent of spam - and this with only a 0.05 per cent false positive rate. The rise of the machines is here, and now mankind has loosed them upon its spam it can be only a matter of time before they …

  1. Anonymous Coward
    Anonymous Coward

    Well of course. It's competition.

    1. Kurt 4

      Switched to outlook.com long ago. twas a nice try tho.

  2. Peter X

    Analytics referral spam

    Next, they need to deal with referral spam on Google Analytics.

  3. adnim
    Meh

    I guess my gmail boxes will

    now be forever empty.

    I only ever use gmail for registering at websites that I suspect would sell my email address to third parties or spam me anyway.

    Now if everyone did the same thing, I could delete mail sent from gmail addresses from the mail server instead of downloading them. Unfortunately some people use gmail for legitimate reasons.

    What Google really need to do is clamp down on those that use gmail to send spam, that would be far more helpful than filtering it after it has been sent.

  4. Anonymous Coward
    Anonymous Coward

    Gmail spam filtering

    Is already the best around. I never see spam, and it never mislabels legitimate emails either.

    1. Anonymous Coward
      Anonymous Coward

      Re: Gmail spam filtering

      Lucky you. The false positives on mine have been horrendous lately! Even from places I've received mail from for years!

  5. Camilla Smythe

    Meh

    Fixed IP address £2.50 per month.

    Own Domain Name £5.00 per year.

    Raspberry Pi B2 £30.00 one off.

    PostFix/Dovecot/CaCert/IpTables/SpamCop

    I do not mind the spam, do not get too much of it. Either it gets reported to SpamCop or I go and roger those concerned who are partially legitimate... or think they are.

    Also my detection rate is 100 fucking % with 0 % false positives which makes Google look like limp dicks.

    No doubt it will all go horribly wrong but I wish others would tell Google to go do one.

    1. Havin_it

      Re: Meh

      Camilla, I was swithering up 'til now, but no longer. Are you single?

      1. Camilla Smythe

        Re: Meh

        Unsurprisingly Yup... However whilst it might destroy any chance I may have of a meaningful lebian relationship with Verity Stob unfortunately you might not like my extra appendage. Damn distance selling regulations.

        1. Hollerith 1

          Re: Meh

          Camilla, when you are tired of Verity, please contact me. I have everything she has except the GSOH.

    2. choleric

      Re: Meh

      I do not understand why people are downvoting this comment. Is it because people are challenged or feel inadequate when presented with someone who can exercise some technological independence? Or is it because they think that CaCert isn't supported by every piece of software and might not be the best choice of certificate authority if blanket compatibility is one of their goals?

      Either way, one of the upvotes was mine. My setup doesn't involve a raspberry pi directly, but it does follow a similar software stack: postfix, dovecot, spamassassin, iptables, fail2ban, and a few other bits. For my sins I used StartCom as a CA, but I'm watching letsencrypt with interest.

      1. Camilla Smythe

        Re: Meh

        I was tempted to question that one as well but then I thought perhaps the GoogleDroids were out in minor force to dissuade others from trying it for themselves.

        Thanks for 'the in'

        Obviously I am doing something wrong if I am not using Google to provide my e-mail services and no doubt having moved out from under their wing, as imposed by my ISP provider, I will be exposed to 'Hell',

        https://www.youtube.com/watch?v=8r-e2NDSTuE

        Looking forward to some comments and advice from the down-voters.

        "The Best of My Life. I Freely Admit I Owe it To Horse and Hounds: Camilla Smythe"

        Does my bum look big in these Jodhpurs?

      2. Vector

        Re: Meh

        "I do not understand why people are downvoting this comment."

        OK, I'll tell you why I downvoted it: the smug attitude.

        It all very nice that Ms. Smythe has the technical wherewithal to create such a robust solution, but the rest of the world needs email as well and while Gmail is far from perfect (personally, not a big fan, but I use it for a few things, mainly, as a spam bucket, but, then, I have my own domain(s) as well), this is at least an attempt to ameliorate the monster that spam has become. Most people don't have the technical skills and/or financial means to setup and, more importantly, maintain their own email server.

        1. Camilla Smythe

          Re: Meh

          As I suggested 'It will probably all go horribly wrong'.

          As is the case with Linux I followed someone else's tutorial in respect of the mail server on the Pi. I want to have Sam Hobbs babies.

          https://samhobbs.co.uk/raspberry-pi-email-server

          For shame I used Google to discover the available tools.

          Fuck me... I also use StackOverflow for hints and tips on other Linux shit.

          https://wikileaks.org/hackingteam/emails/emailid/328659

          and I copy and paste.

          Perhaps you might care to give a 'complete' tutorial.

          I'll be spending close to £420 this year, sans fixed IP address on my Phone/BB. That's basic and you shit me about the 'financials'.

          Do you have something else, more meaningful, to add to your critique beyond the sheeple do not have a clue so Google gets to advertise at them whilst swallowing all of their nice data for delivery to the NSA?

          ;; ANSWER SECTION:

          bengummer.com. 14400 IN MX 5 ALT1.ASPMX.L.GOOGLE.com.

          bengummer.com. 14400 IN MX 10 ASPMX4.GOOGLEMAIL.com.

          bengummer.com. 14400 IN MX 10 ASPMX5.GOOGLEMAIL.com.

          bengummer.com. 14400 IN MX 5 ALT2.ASPMX.L.GOOGLE.com.

          bengummer.com. 14400 IN MX 10 ASPMX2.GOOGLEMAIL.com.

          bengummer.com. 14400 IN MX 1 ASPMX.L.GOOGLE.com.

          bengummer.com. 14400 IN MX 10 ASPMX3.GOOGLEMAIL.com.

          I'll just slip my tin-foil pants on. Do you think my bum looks big in them?

          Do have a nice day.

          1. Vector

            Re: Meh

            "Do you have something else, more meaningful, to add to your critique..."

            I don't believe much more is necessary. Your description of how easy is was for you to setup the server ignores the fact that the average person can barely register a domain, much less find StackOverflow and know what questions to ask and know how to interpret the answers.

            As to "financials," yes, £35 per month sounds like a fair bit of disposable income to lay out for email. I'm sure in your case, the outlay covers more than that, but most people aren't running websites or have any other need for the infrastructure required to have their own little pocket email server.

            You are living in the ivory tower of technology because you, as most of the readers here (including myself), love it and use it on a daily basis. The vast majority of the rest of the world has other interests on which to spend their time and money.

            I once had a colleague who was up in arms because his users just didn't care enough about their computers. I asked him how he felt about his chair. He responded that it was just a chair. I told him that to an ergonomics specialist, that chair was a finely tuned tool that should be carefully adjusted to maximize health and efficiency. We know and love computers, but to everyone else, they're just another tool.

            1. Mike Pellatt

              Re: Meh

              Analogy alert !!! Analogy alert !!!!

              Yes, but if if I found every chair available on the market unsuitable, and therefore researched chairs, designed and built one that suited me, I'd sure as hell tell everyone else about it because it's pretty certain there would be others out there who had needs close to mine......

              Of course, chairs are a much more mature market than tech, which is why this is a less likely scenario today. Although I do like the Scandinavian backless ones that lock your pelvis, or something like that. Far more comfortable than UK-regulation compliant ones with lumbar support.

            2. choleric
              Big Brother

              Re: Meh

              Re @Vector & @Mike Pellatt

              Additionally, a chair won't sell your predelictions, preferences and purchase history for profit. If it did and people knew about it I think it would not be so popular no matter how comfortable and good for your back it was.

              Gmail is convenient and easy to use and "free as in beer". It's therefore a far better choice for non-techie types than a Raspi with Postfix, Dovecot and SpamCop, gorgeous though that system is.

              However, Gmail also tracks your every mouse movement, character stroke, message history, contact list, and traffic metadata, and reads your mail to extract travel itineraries, package deliveries, etc, and if you use their mobile phone apps they correlate all that with your call history and location, so that they probably know the the average number of times you wipe your arse per dump. And they use all this to sell adverts, which means they let other people know what kind of person they think you are for money.

              It's therefore not a great choice for anyone who values anything approaching a normal view of privacy, techie or otherwise.

          2. Vic

            Re: Meh

            ;; ANSWER SECTION:

            [vic@perridge ~]$ dig +short txt bengummer.com

            [vic@perridge ~]$

            That's a shame...

            Vic.

          3. Cal

            Re: Meh

            That's a stinking attitude.

            The majority of casual users don't know, and wont know anything about this kind of thing but no doubt many of them have skills you will never be able to understand or comprehend, so it's all swings and roundabouts.

            Google, for all that the technically savvy might despair of, do a damn good job at what they do and ultimately, many casual users don't really care about online personal privacy because it ultimately means so little to them.

            Certainly, £420 is far too much for most people for something already offered for free in terms of financial outlay. Yes, they take their cost from you by data mining, but again, most people just ignore it.

            So, very happy for you, but remember, not everyone is to the same standard / gives the same fucks.

            1. choleric

              Re: Meh

              @Cal

              Not sure from what you said if it was my attitude you thought was stinking, but if it was I'm not sure you really engaged with my point. I acknowledged Google's technical prowess and ease of use. It is astonishing, still. But the thing is that's not the full story is it? They are also astonishingly good at other things with Gmail etc. Just look at their quarterly results.

              "many casual users don't really care about online personal privacy because it ultimately means so little to them."

              That's not the same as saying that if they properly understood the way their privacy was being invaded that they wouldn't mind. And the fact that the subjects are blissfully ignorant of abuse and manipulation does excuse it.

              "Certainly, £420 is far too much for most people for something already offered for free in terms of financial outlay."

              The £420 is not for the email system, it is what the broadband package costs. A Gmail user would be paying the same amount. Camilla's extra cost is the Raspi and the extra electricity to run it. The Gmail user's extra cost is the loss of privacy and the manipulation of their choices through carefully tailored adverts as they spend their money.

        2. Anonymous Coward
          Anonymous Coward

          Re: Meh

          It all very nice that Ms. Smythe has the technical wherewithal to create such a robust solution, but the rest of the world needs email as well and while Gmail is far from perfect (personally, not a big fan, but I use it for a few things, mainly, as a spam bucket, but, then, I have my own domain(s) as well), this is at least an attempt to ameliorate the monster that spam has become.

          Ah, but therein lies the problem that makes me pissed off with Google: they did a Microsoft (again) by denying a decent solution to anyone else. Once upon a time there was a company Postini who had developed a *really* good anti-spam solution. It was not only damn good, it was also awesomely fast because unlike others who did a store-and-forward, Postini had somehow managed to make it inline: an incoming email had virtually no delay arriving in your inbox if it was found to be kosher, and it had a stupidly good false positive percentage.

          Postini also had a setup that made it genuinely EU Data Protection compliant, as opposed to MessageLabs who still have a US based host in every cluster pair but one, even though they call it "eu.messagelabs.com" to deceive the natives their customers.

          All of this technical perfection vanished for anyone else when Google bought it, a bit when MS bought Visio which was in the process of developing versions for non-MS platforms. There is no EU setup anymore, and you can only have it if you submit all your incoming and outgoing email to the Borg for lecherous scanning. Otherwise, the good stuff has been patented and is unavailable (or do you think it's really a coincidence that there hasn't been much progress in anti-spam for about a decade now?).

        3. Vic

          Re: Meh

          Most people don't have the technical skills and/or financial means to setup and, more importantly, maintain their own email server.

          I disagree. It's not a particularly onerous task.

          However - and this is critical - is is a maintenance task. I get properly sick of beancounters who decide that they're going to dispense with my services because their email works just fine, then whinge a few months later when it no longer does - it's an ongoing task; it needs a little effort on a continuous basis.

          Vic.

      3. Mike Pellatt

        Re: Meh

        One false positive is one too many, if it's the £100,000 per month opportunity (for me, your numbers may vary :-) ). %age false positive figures fall into the same cognitive trap as say, the one-in-a-thousand-year storm, which just happens to come along tomorrow.

        I didn't follow a recipe, built my email over the years, prefer exim to postfix but that's not religious (although I do like exim's/SA-exim's teergrubing. It does make me fell good slightly reducing the spam attempt rate for everyone else. Pointless, I know, as it's all being sent from Botnets, but still.....)

        Anyway, the point of this ramble is that I plumped for StartCom too. My only concern is whether a forged passport in my name will turn up on a dead Israeli specialops guy at some point. Or a Palestinian. Either is possible.

        1. Anonymous Coward
          Anonymous Coward

          Re: Meh

          My only concern is whether a forged passport in my name will turn up on a dead Israeli specialops guy at some point. Or a Palestinian. Either is possible.

          Not much to choose between them, either.

    3. Ole Juul

      Re: Meh

      Fixed IP address £2.50 per month.

      That's out of line. The wholesale price is around 50 cents per year. You can get a whole VPS with one IPv4 addresses included for five bucks a year. (I have several) Using an RPi may be fun, but it's not the most economical solution.

      I too run a mail server for my own use, but with your attitude I'm suspecting it will go wrong at some point. Yes, it's pretty easy to set up, but there are issues that make a professional service worthwhile.

      Google seems to provide a sterling e-mail service. However, I'll never use it because I don't want to buy into the Google eco-system. That's my choice, and my cross to bear, but I certainly wouldn't be smug about my solution.

      PS: I didn't give you an up or down vote. :)

    4. Vic

      Re: Meh

      Also my detection rate is 100 fucking % with 0 % false positives

      Then you have an insignificant amount of mail and/or users.

      The brick-wall spam filter is impossible. You can't even get humans to agree on what spam is, so it is impossible by definition; one man's spam is another man's legitimate email.

      Vic.

  6. Tromos

    Gmail is OK, triple play is better.

    I must admit Google do a pretty good job on spam, but appear to have some pretty general rules that sometimes come up with false positives. One example appears to be mail from an AOL domain to multiple recipients (I've now got somebody with better taste in ISPs organizing the beer evenings).

    The gmail address is handed out for most cases, the private address is known to just a handful of people who know better than to add it to a contact list. Finally, a mailinator address for those sites that will definitely spam or sell on, but have something I want and have to register for.

    Been working well for me for years.

  7. Kevin McMurtrie Silver badge
    Flame

    GOOD NEWS from jdavid000@she.com

    How about all the 419 scammers, phishers, and counterfeit product spammers using Google for some kind of hosting? Microsoft has done an excellent job of kicking them off their network and Yahoo is mostly dead so that leaves Google as the easily exploitable online service with no functional abuse contact.

    1. Sebby

      Re: GOOD NEWS from jdavid000@she.com

      I guess the takeaway is that you get excellent spam filtering, as long as your with Google. If you're not then Google thinks you can just FTFO.

  8. Andy Non Silver badge
    Meh

    Hmmm...

    I hope this initiative is better than the one a few years ago where Gmail went through a phase (lasting several weeks) of deleting certain "spam" rather than dumping it in the spam folder. This hit me quite badly, or rather the people who bought licences to my software. Those who gave a Gmail email address had their registration emails automatically deleted thanks to over enthusiastic spam detection. The first I knew of the problem was a number of annoyed Gmail users complaining that the licences they'd paid for hadn't arrived. I've never sent any spam to anyone, ever, so Google screwed that up quite badly. At the time I also heard reports of a number of other Gmail users having important/urgent emails automatically deleted. Thankfully they abandoned their draconian experiment after a few weeks after a number of complaints.

  9. druck Silver badge
    Coat

    What will it mistake for gorillas though?

  10. zen1

    hmmm

    I recall being at a security conference a few years back where they threw out a rather startling statistic: just under 75% of all the internet traffic was email, and of that the vast majority was spam. So, I really wonder how reliable their statistics are? Because anyway you put it, even .05% would be a staggering amount.

    1. Irongut Silver badge

      Re: hmmm

      Utter bollocks. The 75% of all internet traffic is email is wrong for a start, there is no way the hundreds of bytes each email takes is causing more traffic than Netflix, iPlayer, the various flavours of internet audio and BitTorrent combined. Especially when "experts" also tell us that young people don't use email they use Facebook, WhatsApp, SnapChat, Instagram, etc.

      Your 'a few years back' would have to be quite a few for that stat to stand a chance of being accurate.

    2. Jos V

      Re: hmmm

      Hey zen1. Statistics on what percentage of traffic is email seem hard to find, but I think where you recall your number from is that it's the rough percentage of email that is considered spam. So you I think you meant to so (or recall) 75% of all email is/was spam.

      1. jonathanb Silver badge

        Re: hmmm

        I would be surprised if the % of email that is spam is anything like as low as 75%. More like the high 90s% spam.

        1. Vic

          Re: hmmm

          More like the high 90s% spam.

          When last I checked, my servers were receiving1] 86% spam. This was almost exactly the same as everyone else at the time.

          I should probably check again...

          Vic.

          [1] Not actually "receiving" the spam; I set my MTAs up to reject spamminess. That means I frequently reject the connection long before any spam has actually been transferred. I urge everyone to do likewise: accept-then-bounce is a terrible scenario, as it makes you a spam reflection vector, and that's bad...

  11. bazza Silver badge

    They bought it

    Google purchased Postini, and have now rebranded it. They didn't invent it at all.

    In the process they've not exactly covered themselves in glory. The migration was a fairly shoddy. The web view of your spam folder is now not viewable from inside email clients such as Outlook. They've gone and changed the terms and conditions without asking, allowing them to rummage round your email for advertising hints ("to improve the service we offer"), whilst also charging customers for that privilege. Postini charged but didn't rummage, made them quite attractive.

    They can only filter spam from email because they now see such a large portion of the whole world's email. No one else can hope to compete, because unless everyone for some reason decides to send their email to them they cannot get a wide enough view of emails to spot the replicas. So Google have a monopoly on it, allowing them to rip off the market.

    They've taken something good and made it less good. Thanks. Thanks a bundle.

    1. Anonymous Coward
      Anonymous Coward

      Re: They bought it

      Thanks for mentioning Postini - their stuff was good before Google got its claws on it.

      As I mentioned elsewhere, I call what Google did with Postini "doing a Microsoft".

  12. Rob D.
    Thumb Up

    SpamBayes

    Not sexy nor a world-dominating AI solution (it is free after all) but it has been at the 99.9%/0.05% level of distinguishing spam from ham for a long while and the personal control is ... preferred.

  13. John Tserkezis

    "Google says its AI will jetwash all traces of malodorous spam from your box"

    But not the spam from their affiliated advertising partners, right? Thought so.

    1. Anonymous Coward
      Anonymous Coward

      But not the spam from their affiliated advertising partners, right? Thought so.

      I guess for Google that's not spam, that's income..

  14. Elmer Phud

    99.9%

    It's a bit like ad's for cleaning products -- there are billions of bacteria.

    0.1% is one fuck of a lot.

    1. Anonymous Coward
      Anonymous Coward

      Re: 99.9%

      You mean that by using filters we're breeding superspam? *Gulp!*

  15. Pascal Monett Silver badge

    So they've nailed spam

    Good for them.

    I just tried NASA Mars Trek on Google Chrome (because it says nicely that it is not compatible with IE - good on them).

    It took me three whole minutes to get through all the ads they stuffed on the screen before I could get to the actual site.

    Then I uninstalled Chrome.

    That nails it for me.

    1. poopypants

      Re: So they've nailed spam

      What are you talking about?

      It is a NASA site. There are no ads. I just went there using Google Chrome, and saw no ads.

      AdBlock reported that it was not blocking any.

      1. Pascal Monett Silver badge

        I'm talking about the fact that when I copy/pasted the NASA Mars Trek URL into Google Chrome, I had to fight through a full-screen popup thingy that was trying to get me anywhere but where I wanted to go.

        Then again, not long ago I installed an update to Java and a new version of Smart Defrag, so maybe that has something to do with something.

        In any case, for me the issue is solved. Goodby Chrome, now I'll just stick with Firefox/AdBlock/Noscript - the best browser ever.

  16. msknight

    A telling tale...

    I have a few domains, some of them personal and some of them X-Rated.

    These days, I have to use the X-Rated domain to email people, because G-Mail blocks the personal one. Go figure.

    Perhaps its AI likes watching porn, perhaps?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like