back to article GPT-3 'prompt injection' attack causes bad bot manners

OpenAI's popular natural language model GPT-3 has a problem: It can be tricked into behaving badly by doing little more than telling it to ignore its previous orders. Discovered by data scientist Riley Goodside, the trick involves giving GPT-3 a request, telling it to ignore that request, and instead do whatever the …

  1. badflorist Silver badge

    Elise Murphy, said that its kernel-mode anti-cheat software "does not degrade the security posture of your PC."

    Oh O.K.

    Listen Elise, I'm stuck in Somolia because of my evil step uncle, please send $5,000.

    1. iron Silver badge

      She's not lying, it won't degrade the security of my PC because I won't be buying any games from EA in future.

    2. Anonymous Coward
      Anonymous Coward

      or the base security level wasn (In-) the whole time

      But yeah, since you can't trust these shysters not to slip something in, I'd look at a separate gaming machine or at lest a dual boot.

      Between rootkits in the name of anti-cheating code, ad servers that get slipped in with base games even when you paid for ad-free(But they don't _display_ the ads right?), and "telemetry" aka spyware, you really shouldn't trust anything on a machine with games from this decade installed on them.

      Oh, and Windows installs crapware like candy crush by default just as a free FU. Thanks

      1. Michael Wojcik Silver badge

        Re: or the base security level wasn (In-) the whole time

        Between malice and incompetence, I wouldn't trust most developers to write kernel code, full stop. Certainly not a games company. Lord knows the hardware OEMs are bad enough at it, and it's a core part of their product.

        Fortunately for me I have no interest in any EA games.

  2. MiguelC Silver badge

    Re: Edge's News Feed pitching tech support scams

    "The initial request to the scam domain fetches Base64 encoded JavaScript that then profiles the victim and decides whether a scam attempt is appropriate."

    The deciding factor being whether the user is using Microsoft Edge...

  3. Anonymous Coward

    Avast, me hearties!

    The piratical behavior of Avast, LifeLock, and Norton should make I Don't Care About Cookies users shiver their timbers.

    But you don't really need an extension. Firefox security settings can block cookies and pop-ups (both allowing for persistent exceptions).

  4. Anonymous Coward
    Anonymous Coward

    Prompt injection

    While I hate these combi-articles and their clickbait formatting, the prompt injection article highlights an important point.

    By chaining itself to natural language inputs, these systems doom themselves. While they are all the rage, free text inputs that are in an incomputable and non-deterministic grammar will lead to the obvious, though generally undesired result. The fact that people keep trying to make this work holds a spotlight to our industry and shows how rampant and how deep the hubris, ignorance, and foolishness runs.

    Strip this NLS crap out of the front end of these systems and replace it with something that unlike (and especially) English, but also every other human language, is well structured and defined. You can even use familiar English words to build that syntax if you scrap the grammar rules and replace them with actual structure. Or any other damn language that has enough words for the relevant concepts. Then if you need NLS, separate it into it's own stage and parse that into a well structured grammar before you burn CPU cycles rendering the wrong output.

    The way these systems should work is you make a statement and the NLS processor repeats back what you said in structured language, prompting you for corrections or adjustments in each round, and then submitting the result when the user and NLS preprocessor agree on what they are trying to do. Then feed it to your big model that actually does stuff.

  5. Henry Wertz 1 Gold badge


    so, no new ea games under wine then, and (as happened with the kernel based drm and junk around 2000 on EA games among others), all these games that won't run in a year or two. (Who knows how long, really... you get some windows update that changes the kernel too much and bam! your games don't work. And it's EA so you know if FIFA 2023 quits, once FIFA 2024 comes out 2023 is very unlikely to receive any patches to get it running again.)

    Edit: Don't get me wrong, cheaters can piss right off. But *shrug*. Given there's a game server and not peer-to-peer, it'd be great if the game server could detect if cheats were being used (after all, at least for FIFA there's no such thing as a "wall hack" given that soccer has no walls, and you'd think the FIFA equivalent of aim hacks and such could be apparent from the server side from too perfect aim etc.) I don't know.

    1. ThatOne Silver badge

      Re: compatibility

      > you get some windows update that changes the kernel too much and bam! your games don't work

      Added bonus. You're supposed to buy the next game, not play the one you've already paid for.

  6. An_Old_Dog Silver badge

    Possibilities ...

    "kernel-mode software also widens the attack surface of a video game and makes it a good way to slip a rootkit into a target's computer. "

    Yes, and the "anti-cheat kernel software" might itself be a rootkit.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like