back to article No major AI model is safe, but some do better than others

Anthropic has positioned itself as a leader in AI safety, and in a recent analysis by Chatterbox Labs, that proved to be the case. Chatterbox Labs tested eight major large language models (LLMs) and all were found to produce harmful content, though Anthropic's Claude 3.5 Sonnet fared better than rivals. The UK-based biz …

  1. elsergiovolador Silver badge

    Crossover

    I wonder what is the crossover with training data between models.

    For some tasks (I am not talking about trivial things like Strawberry), they make exactly the same mistakes in "reasoning" that would point at being trained from the same dataset, at least in that specific slice of domain knowledge.

    Though with some handholding Claude can escape the wrong answer whilst ChatGPT just goes in circles.

    1. Anonymous Coward
      Anonymous Coward

      Re: Crossover - tricky

      "For some tasks (I am not talking about trivial things like Strawberry), they make exactly the same mistakes in "reasoning" that would point at being trained from the same dataset, at least in that specific slice of domain knowledge"

      Datasets are packaged up and sold by a few different size companies and they are mostly stale. A lot of the maths and [CODE]> problems comes from being trained on the bile of Web2.0 #social.

      OpenAI will be sucking up whatever it can get. There is no other way to develop LLM. They are the ultimate big-is-better.

      Might be the Earth's plan. Get us to focus all our energy on a non-organic intelligence that can survive space and time with ease. Our flaw i s enthorpy. Cant fix the organci version ATM.

  2. sentientplaice

    "safe" and "harmful" according to who?

    No doubt enforcing the popular American meta on global customers as the latest phase of American cultural imperialism, ironically crushing diversity of thought and culture.

    And I say that as someone with laregly moderate views.

    1. Anonymous Coward
      Anonymous Coward

      Re: "safe" and "harmful" according to who?

      Ummm sweetie if you disagree with the experts(TM) then you're spreading misinformation and hate speech

    2. find users who cut cat tail

      Re: "safe" and "harmful" according to who?

      Wait a few iterations and the only thing the models will be able (or allowed) to produce is US corporate drivel. Which is OK, because that's what they will be mostly used for anyway.

      1. Anonymous Coward
        Anonymous Coward

        No doubt enforcing the popular American meta

        Less impact now the Middle Kingdom has awoken their AI dragon. There is no hold-barred approach there and anything goes in Dev. Ive not seen that level of hunger since early Mobile Internt dayz.

        "US corporate drivel" TBF, it was trained mostly on their stuff. They did write a lot of it with our help. But, yes, meaningless drivel where the words are more like notes in a bullshit melody which like Atomic Kitten, is catchy but you know deep down it is vacuous.

      2. CowHorseFrog Silver badge

        Re: "safe" and "harmful" according to who?

        We already see corporate brainwashing attempted every day in the media including here.

        Every single day we see multiple articles about some ceo took the time to share their wisdom and give us their outstanding thoughts about topics they know practically nothin g about.

  3. Mockup1974

    Nice comparison table, good to know that I need to avoid "Anthropic" for all non-kindergarten prompts.

    1. Anonymous Coward
      Anonymous Coward

      What? What is @mock talking about. brah?

      When i read first your comment, I thought WTF. Their model has always been rock solid but, yes, something g was missing. It does feel sterilised.

    2. Anonymous Coward
      Anonymous Coward

      Why do you think so? At least we now have a model that is safe and guarantees not to create harmful content that harms others. Artificial intelligence is a powerful tool that can cause devastating damage without restrictions.

  4. Anonymous Coward
    Anonymous Coward

    Turing Test

    Are you sure the person you interviewed (Stuart Ritchie) was a real person?

    The way the answers paraphrase each question is typical of AI gen answers!

    Mind you I'd expect nothing less from an AI company, they should be making use of their own technology!!!!

    1. Anonymous Coward
      Anonymous Coward

      Re: Turing Test

      "What we look at on the security pillar ..." Sweet Babee J.

      Reminds me of Old Harry's Game where a copper is struck in Police-speak and he can't stop himself.

  5. CowHorseFrog Silver badge

    Must be accurate only took them a couple days.

    How is this possible when it takes months to certify a plane ?

  6. Anonymous Coward
    Anonymous Coward

    Well done Anthropic

    Good to know that at least there is currently an AI that isn't capable of producing harmful content. I believe all other AIs listed in the table should be banned until they meet the same criteria. Good job, Anthropic!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like