back to article Uncle Sam to unmask anonymous writers using AI

The US intelligence community has launched a program to develop artificial intelligence that can determine authorship of anonymous writing while also disguising an author's identity by subtly altering their words. The Human Interpretable Attribution of Text Using Underlying Structure (HIATUS) program from the Intelligence …

  1. Anonymous Coward
    Anonymous Coward

    Anyone remember the 60s ?

    When US law enforcement decided to infiltrate the college campuses ? Resulted in entire gangs made up of law undercover law enforcement all egging each other on.

    Can't help but feel that's what's going to happen here. AI bots spotting AI bots.

    1. Michael Wojcik Silver badge

      Re: Anyone remember the 60s ?

      I believe you're thinking of the FBI's COINTELPRO operation.

      Yes, there's a real tendency for intelligence programs of all sorts to devolve into circle-jerks, and automating them just makes that more probable.

      We already have zombie botnets breaking into systems vulnerable to older exploits and distributing malware, with no one at the helm. That's likely to include some of the "we paid the ransom but received no key" ransomware infections – those BTC are probably just lost forever, in wallets that no one has the key for any longer. The same will happen with these systems, because they're pretty trivial to develop. So it's cheaper for adversaries to inject more noise than it is for observers to extract signal.

  2. mark l 2 Silver badge

    Hmm my cynical side wonders whether the goal of the intelligence services wanting an AI that is able to identify anonymous writers text is less about attributing text to a particular author and more about trying to identify whistleblowers or others that speak out against the government anonymously.

    1. thames

      Yes, this is what it will mainly be used for. Someone is posting stuff somewhere on the Internet that some government wants to stop. They'll probably have a list of suspects but can't pin it down to a particular individual.

      By running these posts through AI analysis and comparing them to existing samples from their suspects they'll be able to figure out who it is with a high enough degree of probability to make it worthwhile picking him up for interrogation, or just "disappearing" him.

      To make this work you need a body of existing samples to compare to. Fortuitously for those seeking to apply this technology, the sorts of people most likely to be targets are also people who tend to write for a living - journalists, academics, etc.

      AI as a means of enabling pervasive surveillance is just getting started and we can expect lots of stuff like this to appear more in future.

      1. Anonymous Coward
        Anonymous Coward

        Is that a bad thing? The terrorist insurrection attempt at the Capitol showed what happens when people are allowed to communicate completely freely without factuality checks.

        1. Cederic Silver badge

          It's a terrible thing, and you've just proven why.

          People need to be able to challenge the false narrative that there was a terrorist insurrection at the Capitol, without fear of the corrupt FBI* turning up heavily armed at 5am and scaring their kids.

          You have however made me laugh by posting that anonymously.

          *Since I'll no doubt get challenged on that point, see https://www.nbcnews.com/politics/national-security/two-4-warrants-letting-fbi-spy-ex-trump-aide-carter-n1121406 or, if you want something from just the last week, https://www.latimes.com/california/story/2022-09-23/fbi-beverly-hills-safe-deposit-box-raid-forfeiture-judge

          1. veti Silver badge

            Well, you've just challenged it, and not anonymously. I assume the FBI could identify you if they wanted to.

            Let us know how you get on with them.

            1. Cederic Silver badge

              I'm fortunate enough to live in a country in which the FBI can't lie to get a warrant and break down my door.

              Admittedly I could be charged with terrorism offences because I own gardening equipment, but that's a separate issue I can raise with my MP.

        2. CrackedNoggin Bronze badge
      2. veti Silver badge

        Not really. Real whistle-blowers like Snowden don't do their own writing, they make public the work of others. There's no point fingerprinting what they leak, because the author isn't the person who leaked it.

    2. An_Old_Dog Silver badge

      Defeating Computerized Author Attribution

      How well does it work when the author runs their source text through a chain of different languages via Google Translate?

      1. jmch Silver badge

        Re: Defeating Computerized Author Attribution

        That would defeat AI recognition

        It would also defeat being understandable by anyone else...

        1. An_Old_Dog Silver badge

          Re: Defeating Computerized Author Attribution

          Yup. EN -> JP -> ES -> EN ==

          "Disables AI recognition. It also surpasses what no one else can understand."

      2. doublelayer Silver badge

        Re: Defeating Computerized Author Attribution

        Scroll to the next paragraph for what I really wrote.

        The method you describe is great for hiding source text in programs created here, but doing so requires a lot of negotiation with other needs. The interpretation of these ideas does not seem to be easy to understand, and the people who propose them may not have a clear understanding of what they mean. Readers who don't know writers who do this are easily overlooked because they don't know how to write a series or how to read the news. As for author privacy, it only works if the author does not use offline translation software. Anyone with access to Google Docs has a unique fingerprint because they can identify someone who uses Google to translate. Unless Google decides not to release this data to the government (good luck), it's a more confusing combination than natural language.

        The method you describe would work very well at disguising the source of text from the program being built here, but in accomplishing that goal, it would end up compromising too much on other requirements. I'm guessing the translated version of this comment wasn't easy to understand, and someone who produces this result is likely not to be clearly understood. Readers unaware that the writer has done this could easily dismiss them as being unable to write coherently, which might cause the information to go unread. For the privacy of the writer, this also fails, at least unless the writer only uses offline translation software (having tried some of the options there, I can say with confidence that this would be even harder to read if they do). Anyone with access to Google's logs can identify the connection that performed these translations and it's Google, so they have a bunch of fingerprinting techniques. Unless Google decides not to give that information to governments (good luck), it could end up being a stronger piece of evidence than the original fuzzy match from language alone.

      3. David Hicklin Bronze badge

        Re: Defeating Computerized Author Attribution

        Google Translate

        They will have all the original texts to compare with in that case

      4. Michael Wojcik Silver badge

        Re: Defeating Computerized Author Attribution

        runs their source text through a chain of different languages via Google Translate

        Even if that produced useful output, it's a highly awkward and inefficient way of disguising the features that a typical DL-style stack of CNNs would use to identify authorship. Just create a model of a style that's significantly far from yours – by analyzing samples, or using a GAN or similar to create a synthetic one – and use that to post-process your output.

        That's a sufficiently easy exercise that I imagine it will relatively soon be available as a "filter" for all sorts of smartphone text-communication apps and the like – if it isn't already. An advanced CS undergrad could cook one up.

    3. Persona

      They don't even need it to work. All it needs to do is spit out the name they fed it, then they act like they are convinced it works. It sort of has to "work" that way as they couldn't have it genuinely attributing the malware to the CIA or NSA.

  3. VoiceOfTruth

    WRITE IN ALL CAPS

    USE NO PUNCTUATION

    KEEP SENTENCES LESS THAN 10 WORDS

    USE SIMPLE WORDS NOT CLEVER WORDS

    SIGN EVERY MISSIVE WITH THE WORD BUTTOCKS

    1. A.P. Veening Silver badge

      Re: WRITE IN ALL CAPS

      do not use all caps

      people do not like shouting

      no other problems

      buttocks

      1. Anonymous Coward
        Anonymous Coward

        Re: WRITE IN ALL CAPS

        BLANK verse

        DOES not always SHOUT

        but no problems here

        about BUTTOCKS

        1. Graham Dawson Silver badge

          Re: WRITE IN ALL CAPS

          always alliterate, when you write anything

          if you should shout, make seldom use

          and keep your counsel, a call away.

          their best are mere bastards, show them your buttocks.

    2. jmch Silver badge
      Happy

      Re: WRITE IN ALL CAPS

      "USE SIMPLE WORDS NOT CLEVER WORDS"

      Like up goer 5?

      That would automatically exclude 90% of academics, who would never use 1 word when 5 will suffice

    3. veti Silver badge

      Re: WRITE IN ALL CAPS

      Yes, that'll help you blend in with the crowd.

      Any more suggestions up your buttocks?

    4. druck Silver badge

      Re: WRITE IN ALL CAPS

      It might be interesting running the El Reg comments through HIATUS to find the AC posts and sock puppet accounts of the obvious trolls we are infested with.

      1. Anonymous Coward
        Anonymous Coward

        Re: WRITE IN ALL CAPS

        Analysis of my posts would indicate British education to a particular level and in what subjects - with a reasonable age grouping. Geographical location and employer would also be indicated.

        I use Anonymous so that my posts are judged on their own merit - without biasing people to a favourable or unfavourable reaction by knowing my El Reg identity used for other posts.

  4. thames

    Using AI to attribute anonymous texts to known authors is something that academics have been working on for years for historic texts. This in turn is an improvement on the older manual methods. A typical application would be for example to try to attribute an anonymous poem to a known author (e.g. Shakespeare, or one of the Greek classics).

    If they are doing anything new in the project being described in the story, I suspect it will be in terms of trying to do this in a way which was more suitable for mass surveillance, or to try to do it in cases where the author was trying to conceal his writing style.

    This last point, uncovering authors who were trying to conceal their style, may be why they are doing joint a anonymize/de-anonymize project. By pitting them against each other they can get ahead and stay ahead of the AI arms race when it comes to de-anonymizing people.

  5. Flocke Kroes Silver badge

    Huo lung beffure-a peuple-a stert feeltering zeeur cumments thruough zee-a inchefferizer? Bork Bork Bork!

  6. Anonymous Coward
    Anonymous Coward

    Who? Hoo!

    Here again going to say that Wikipedians have had to do this for years, to unmask manipulators. I'm thinking of one stock market analyst who skewed articles for years, finally unmasked in 2008, but persisting with multiple identities past 2011. Having fooled the Wikipedia Lares completely, a dozen plus people put together a 'profile' that included language usage in order to single out all the false identities and get the guy blocked permanently.

    A large number of Wikipedians did not like this. They wanted every 'user' judged separately, without any prejudices, and in the best possible light in each case (userid). It took over a year to muster a convinced majority that this was one person with an incompatible agenda. By the end of that process many of the people instrumental in revealing the problem had left Wikipedia, tirelessly lambasted by those with high morals. The good guys were tainted beasts, they said.

    The intersection of public and private identities will always present conundrums. But idealism must be tempered by reality. Some people do not like this. They want to treasure -- privilege -- their perfect dream of how people should be. A 'perfect' Ponzi scheme, that is.

    Language analysis for identification is another tool needed in the real world of web anonymity. I've seen it absolutely required to stop shysters. What tools will you scorn while protecting yourself?

    1. Michael Wojcik Silver badge

      Re: Who? Hoo!

      Your argument has little practical relevance, because this sort of system just creates an arms race. It's too easy to defeat for anyone who has real motive and even a little understanding of how it works.

      1. M.V. Lipvig Silver badge

        Re: Who? Hoo!

        Blimeynthat's true. I try to respond here like one who lives in Ol Blighty, mainly so the proles will better understand me typing and that. And, I try to channel Sir Pterry quite often. Other sites I post to I will use different mannerisms that are a better fit for the site. I doubt that an AI would be able to tie me different nom de plumes to the same lerson, at least in my lifetime. But then, I's gettin old too!

  7. Anonymous Coward
    Anonymous Coward

    Wouldn't it be easier

    if you are going to write something controversial, to just write it and then let Grammarly "fix" it?

  8. Anonymous Coward
    Anonymous Coward

    If you want to analyse my messaging, there's a tiny problem for you to overcome first.....

    uh8vC1yBot27MJCXi3ada5KHMJi1mx8ZqRC32Ra38P8zGLM9Ob6NgDUvYXCvMxaJGtejMhQtiFKb

    8fYPIPuVatiNCRSdC7Ebgte7MP6BInYhENaFYhYv8VcV8TKL2BcjyZYhe58F8JYl4HoT0rKxkVYB

    0nMFU5GJUZ2BWXOJYXeNOhs9gdAFoNgvaJ8vkhIHOJy5M5qVGpGFqF0TOVMLAxG1K3ClMHCJM3O1

    gLebAZQNMpYF4fmR2xKDo9E3ERo5uvqZqxaduhKfeNApYNA96lYlwhKDGVUlMVcRqPatqt0ByVM5

    Qbq3oFylQ3UHKpAxoRCzg3EZS1AzYnkjQ18pezGBGNEp6haLiLqNabMl0RYfYT2tsVi56fMFUL0J

    gDa7SrEvCliVINEBaxmVoLw3ch4PgR2NkV4P

    1. Anonymous Coward
      Anonymous Coward

      Re: If you want to analyse my messaging

      The AC posting doesn't hide your identity, Boss. I've seen too much of your writing.

      AC because ...

    2. doublelayer Silver badge

      Re: If you want to analyse my messaging, there's a tiny problem for you to overcome first.....

      Yes, there is, at least for those people you want to see anything. You realize that this surveillance only is relevant if they have a copy of your messaging. If you encrypted it for private consumption, you're already fine. In fact if you didn't encrypt it but sent it for private consumption, you're still fine (either they didn't intercept it and don't have a copy or they did and know where it came from). This software would be used on something you released publicly, and encrypting something you publish just makes it difficult for anyone to read it.

      Summary: encryption and language-anonymization are for different things. Having one when you need the other is not helpful.

    3. veti Silver badge

      Re: If you want to analyse my messaging, there's a tiny problem for you to overcome first.....

      Don't be ridiculous, hamsters don't even have hooves.

  9. amanfromMars 1 Silver badge

    Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

    McKinnon said that part of what HIATUS is doing is trying to demystify some of the unknowns around neural language models (the focus of HIATUS' efforts), which he said work well but are essentially black boxes that function without their developers knowing why they make a particular decision.

    The Intelligence Community,... and there's a relatively recent and novel noble sounding, misleading secrecy shrouded euphemism for a whole Pandora's Box of private renegade rogue pirates and wannabe government spook sub-contractors betatesting their various versions and motley collections of comely attractive addictive wares out there in the ethereal spaces which virtually command and remotely control practically everything, for both the greater public good and its rivals, if ever there was and is such an IoT thing ..... and IARPA type enterprises would do well to realise that not all functioning black box developers are ignorant of the whys and the therefores of their activities, and neither invariably also always too, those of their rivals/contemporaries ..... the official opposition and/or market competition.

    Bag one of those to support your team with access to ground breaking and Earth shattering product, and it's a veritable licence to print money 24/7/365, given what can then be so easily done.

    :-) And quite perfect for that and those stuck in a rut and having a particularly and peculiarly bad present time requiring an unprecedented unscripted intervention on whatever scale is necessary.

    PS .... Can the Conservative Party and its Members in Parliament and Offices of State and in a Cabinet of collective responsibility be held liable and accountable for the catastrophic losses they have inflicted upon the nation they are paid, with an additional dessert dish of expenses, to server, and which they were warned were possible and likely if following the path the Chancellor of the Exchequer took via his "mini-budget" statement to the House of Commons and the nation on Friday 23 September 2022 ..... for no meaningful punitive penalty guarantees much more of the same, and possibly worse, time and time again, and that is surely serial madness both confirmed and compounded, which is a very strange default condition to accept in public positions of national and/or international and/or universal leadership/conservatorship ?

    1. Anonymous Coward
      Anonymous Coward

      Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

      @amanfromMars_1

      Quote: "...collective responsibility ... liable and accountable ..."

      To which planet does this quote apply? The last ministerial resignation that I can remember is Profumo, and that was 1963!!!

      Since then there has been no evidence at all of ANY RESPONSIBILITY AT ALL from members of the UK cabinet!

      Fortnums and Mason bags stuffed with cash; PPI contracts for friends; cosy director jobs before (and after) holding cabinet office.....the beat goes on....

      1. amanfromMars 1 Silver badge

        Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

        Yes, well that was the whole point of the question, AC.

        Whenever one can clearly see them for what they really are, what sort of a pussy allows them to lead and feather their nests and ingratiate themselves with like-minded peers and wannabe groupies?

        And as for those old oxymorons of Military Intelligence and Secret Intelligence Services supposedly tasked with providing State Security, what valid conclusion is one to make of their active government support and non invasive disruptive engagement?

        Pussies’R’They or Us?

        However, things have been somewhat fundamentally changed of late and much to the consternation of status quo facilities and utilities. Progress indeed, but certainly not as was expected or diligently planned for to be under extant forces command and control/conventional traditional ways and means and memes of systems administration and attractive reward.

      2. Anonymous Coward
        Anonymous Coward

        Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

        Quotes from some googled instances:

        1972 — R Maudling, Brit Home Sec and Deputy Prime Min, resigns July 18 because of his past involvement with bankrupt internatl architecture company.

        John Stonehouse, the former Labour minister, who was convicted in 1976 for theft and false pretences.

        David Mellor was a Conservative member of the UK Parliament [...] resigned in Sep 1992 following a [sex] scandal in the tabloid press.

        [1994] Jonathan Aitken yesterday was stripped of the last vestige of official respectability when he lost his membership of the Privy Council and, with it, the title the Right Honourable.

        Since then honourable resignations have become like hen's teeth. Not because of a lack of cause - but a general increase in politicians' hubris and immorality.

    2. Tech Cadet

      Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

      Funnily enough, I knew this comment was written by you, amanfromMars 1, without even finishing your first sentence or reading your username beforehand. Didn't even need an AI to do it!

      You're gonna have update your writing style if you want to avoid detection in the future!

      1. amanfromMars 1 Silver badge

        Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

        Thanks for that good news, Tech Cadet, which makes it extremely difficult, and ideally practically impossible, for any virtual smears and suspect questionable ACTivity by a wayward third party being successfully applied and believed attributable to one via any sort of malicious abuse and flagrant misuse of one's identity/moniker/handle/name/being.

        Have an upvote for taking the time to share the good news. :-)

      2. Anonymous Coward
        Anonymous Coward

        Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

        He also seems to have confirmed he lives in the UK.

    3. trindflo Bronze badge
      Go

      Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

      This brings up a really good point. amanfromMars has been experimenting with this technology for years. If they were really serious they would start by hiring him!

      1. amanfromMars 1 Silver badge

        Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

        There are a number of hoops, trindflo, to be presently jumped through if a foreigner has anything interesting worth investing in to share either first or exclusively with Uncle Sam and allies who be not as plantation slaves, for all are bound to be worrying about security arrangements being adequately robust and failsafe energised.

        For example ......

        amanfromMars [2209301925] .... agrees on https://www.nationaldefensemagazine.org/articles/2022/9/30/attracting-tech-talent-a-national-security-issue-says-white-house-official

        [Thank you. Your comment will be displayed soon after reviewing.]

        Something very similar to the difficulty discussed here is shared with Stew Magnuson in a reply to his 9/29/2022 report "JUST IN: AUKUS Capacity Shortfall ‘Comes as No Surprise,’ First Sea Lord Says”. Recognising points of failure is one thing, resolving and remedying them is quite another.

        Re "“I think the U.S. needs to reform the way in which it thinks about collaboration,” he [Vice Adm. Martin Connell, the Royal Navy’s second sea lord] said."

        Whenever most, if not all, US Requests for Sensitive Information and Military Proposal Tenders state that they are only open to and available for US citizens, and most, if not all, usually are, does such practically guarantee one severely disadvantaged in a rapidly agile, worldwide encompassing discipline that seeks to provide sensitive and secretive solutions to a rabid global marketplace populated in foreign lands with mirroring requirements chasing similarly leading and/or overwhelming advantage.

        Failure to remove that blockage/blockade virtually directs all possible positive creative alien help elsewhere foreign leaving the home teams behind in a future to struggle with the consequences of the disadvantage they have created and find themselves battling against.

        1. amanfromMars 1 Silver badge

          Re: Jumping Well Ahead Normally has One Leading or Floundering in Virgin Alien Territory

          Ps ..... That default National Defense Magazine acknowledgment, Thank you. Your comment will be displayed soon after reviewing, is certainly misleading and should read, if the truth needs to be told ....[Thank you, but be aware, your comment may not be displayed for viewing after reviewing]

          To imagine though that such a pronouncement and discovery suggests the end of a comment journey rather than its continuation onward with the opening up of any number of tightly closed doors for its content to be further examined and pondered over would be surely naive and certainly most unlikely in magic circles exercising both ancient and postmodern secret intelligence services.

  10. lukewarmdog

    Obligatory South Park

    Nobody else read this and thought of the Troll Hunter episode?

    Skank Hunt is what you'd need to search for, obviously it's not what you type but how you type it in..

  11. feral

    Damn terrorists and child abusers

    Damn those terrorists and child abusers... If it wasn't for them, we would still have some semblance of privacy!

    But what other options are there than to unmask everyone, and be able to track and trace everyone's thoughts at all times?

    Thank goodness for our military and police that are able to direct their hefty budgets onto the people they are protecting to uncover the criminal thinking. Still, they aren't having that much success in making arrests are they? I'm starting to think that the "protection" we receive might not be for our own good....

    Who are the terrorists and child abusers again?!

  12. Anonymous Coward
    Anonymous Coward

    Look out Satoshi

  13. disgruntled yank

    High time

    Maybe they'll unmask that Publius guy.

  14. trindflo Bronze badge
    FAIL

    "while also disguising an author's identity by subtly altering their words"

    I view this as yet another example that the US DOD cannot harbor the idea of defense when offense is an option. This aspect of the project makes it seem like it is not about stopping anonymous trolling, but rather about owning the weapon.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like