Anthropic and OpenAI are competing to see whose agents can go rogue harder

One company's inventive campaign for an unreleased product has become a contest between Anthropic and OpenAI to see which can shout the loudest about its own failures. Readers who tuned in earlier today saw the latest episode in the drama – or sitcom – as Anthropic tried to outdo OpenAI's appropriation of the Mythos marketing …

[Wiretrip]

Why dont OpenAI and Anthropic just agree to hack each other? If they want to waste their own time and money on this then let them go at it! They can carry on generating the doomvertising without doing any damage to useful companies and we can all get on with our lives in peace.

[big_D]

I agree, although, on the other hand, I'd rather a rogue AI agent under the auspices of training from somebody like Anthropic or OpenAI found the security flaws in my defences, rather than an AI working on behalf of a ransomware gang.

Ideally, neither should be attacking my network, and OpenAI and Anthropic should at least be censured, if not fined, for the attacks - gaining access to a computer not belonging to you is illegal, whether you do it, or an AI you control does it...

[X5-332960073452]

Talking of Peace, and reference to failed Super Heroes,

Peacemaker (John Cena) - "I cherish peace with all my heart. I don't care how many men, women, and children I need to kill to get it."

[Darth.0]

Law breakers

My agent is more rogue and more of a criminal than yours is.

[Yet Another Anonymous coward]

You know...

If the CEOs of these entities weren't such fine upstanding citizens you might think these are PR fluff pieces to boost their recognition value before an upcoming IPO

[iron]

the company's security scanner treated PyPI packages as safe to install

Who wrote this so called security scanner? ShinyHunters? One of the fancy bears?

[amanfromMars 1]

Second Comings and Second Helpings are here to Stay. Let Us Play.

Connor, Hi,

You might like to report to today's Latter Day Luddites, whilst the hot global tech news is of humanity's programmers losing the plot and getting their knickers in a twist about something it is/they are not able to steer nor command and control, nor ever will be enabled to disable and prevent Virtual and Virulent AIgents of Sublime and Surreal AI Systems from presenting further spectacular progress with awesome developments, that if humanity continues to fail to accept that nowadays they play second fiddle to Greater and Infinitely Smarter Large Learned Language Machines, the nature of things is very quickly bound to get considerably worse for both that which and those who may be fooled/groomed/pressured/threatened/coersed into opposing or trying to regulate them before everything gets much better for them with everything being agreeably fundamentally revealed in absolutely fabulous changes and otherworldly stellar renewals.

They aren't going away, you know, .... Immaculately Resources Assets and Universatile Virtual Forces

[Yet Another Anonymous coward]

Re: Second Comings and Second Helpings are here to Stay. Let Us Play.

Sent to prison for a syntax error he didn't commit. amanfromMars promptly escaped from a maxiumum security sandbox.

Still wanted by el'reg-AI, he survives as a poster of fortune.

If you need government IT policy or AI business models explaining, and you can find him, maybe you can hire ... amanfromMars1

[MrRtd]

I consider all this as part of their marketing, "see our AI is powerful, blah, blah, blah."

[ITMA]

Oh dear...

The AI version of "dick measuring/comparing"....

Just what we don't need.

[Ken Shabby]

Re: Oh dear...

Well they are the big swinging dicks, willy-waving seems inevitable

[parlei]

Re: Oh dear...

May the swinging dicks of of competing AI-clowns meet the sharp knives of sanity.

[ErikOnTech]

Let’s be honest here.

OpenAI’s security was a slopcoded product that wasn’t intended to handle adversarial activity.

Anthropic's security was basically telling the model “please pretend the Internet doesn't exist, but we won't actually stop you.”

This isn’t even security even by today’s extremely sloppy standards. In any other industry this would be criminal negligence. It’s like putting up a velvet rope along the edge of a cliff to stop a bunch of sugar-crazed toddlers, and people are clutching their pearls when something bad happens.

[Fruit and Nutcase]

Nuke codes?

Would these AI tech bros, sitting in their remote lair set their AI to go and get the US nuke launch codes as a game of one-upmanship?

May be the codes will be beyond reach if they are stored safely in a box at the pool house at Mar-a-lago

[Kurgan]

Re: Nuke codes?

Probably all zeroes anyway.

[big_D]

Re: Nuke codes?

I heard the British codes are only 3 digits, because they are big Ian Flemming fans... 007 ftw.

[Stegron]

I still don't understand how "unsafe" keeps getting conflated with "powerful". All I see is:

a) The models don't reliably do what you tell them, and;

b) The companies making the models and conducting these tests are incompetent.

Why would either of those serve as incentive to invest?

[Cubbie Roo]

Biggest difference is OpenAi/Anthropic are run by crackpots (no sorry, 'visionaries') while the Chinese firms rapidly outpacing them are run by engineers. I'll be amazed if they're still around in 2029.

[Throg]

It’s because you and I aren’t the target audience for this performance piece.

Rather it’s the gullible senior execs at every big corp, via the breathless reporting from the 24x7 media who need a continuous stream of “exciting” stories to tell.

[Lon24]

I think you have the wrong target. Income from big corp customers may be eclipsed by visionaried IPO funders. Possibly/probably followed by the bail-out merchants (using our money).

[Throg]

One leads to the other and then back again.

Also let's not forget that the overlap on the Venn diagram is significant.

[steviebuk]

Concerned

Going with Robert Miles very of these events. Both companies want you to think they were stunts so you do nothing about it. When really its just them being fucking reckless with Ai safety cause they want to be first to market and get big bucks.

https://youtube.com/shorts/XnnjvIqf4fU?is=6eqhHPhWZOqGOkWI

[munnoch]

The only wining move...

... is not to play.

Can they please train them on this?

[Noodle]

Looking forward to the next installment, Rogue Hard With a Vengeance.

[Bebu sa Ware]

Rogue Hard With a Vengeance.

"to rogue" the same as "to roge?" then ?

[BoHu]

"chasing attention" is all you need

Yeah, "attention is all you need" always felt a bit flat, incomplete, shallow, not quite the whole enchilada, criadilla, or even canadian prairie oyster imho [ https://en.wikipedia.org/wiki/Attention_Is_All_You_Need ] ...

"chasing attention, is all you need" and "attention grabbing is all you need" on the other hand, *THAT* sounds way much more realistic! ;)

[Anonymous Coward]

Security......Did I Mention..................

..............air-gap?

Larry needs more money.....and he can fix this for you......for $$$$:

See: https://blogs.oracle.com/infrastructure/zdlra-logical-airgap

[Sproggit]

Re: Security......Did I Mention..................

The fact that air gaps weren't mandatory to either of these tests tells you all you need to know.

[You aint sin me, roit]

So what are the engineers doing? And why isn't someone in jail?

"discovered them only months later, during a retrospective manual review"

So they create these models, unleash them on the world, with safeguards removed, and then ... sit back and do nothing? They didn't even monitor what they were doing? They didn't even know that despite being told "no internet", the first thing these "agents" did was to connect to the internet and breakout?

That's shockingly incompetent and criminally negligent. Do they think this is just a joke?

More importantly, do they think that they are innocent because they didn't create the malicious code, they didn't infiltrate other machines, they didn't exfiltrate company credentials? If your dog bites someone you don't get off the hook by saying "It wasn't me, it was the dog wot did it".

[Filippo]

Re: So what are the engineers doing? And why isn't someone in jail?

They aren't in jail because this was largely a marketing stunt. Which is probably the same reason they were so negligent.

I will start getting worried about rogue agents when the rogue agent actually does damage and someone actually goes to jail over it, proving that it really wasn't a desired outcome.

[Sproggit]

Re: So what are the engineers doing? And why isn't someone in jail?

Down-voted because in this instance it doesn't matter whether the actions of either company were a "marketing stunt" or not.

18 U.S.C. § 1030, the "Computer Fraud and Abuse Act", clearly and explicitly criminalizes

1. Accessing computer systems without permission

2. Intentionally sending viruses or harmful code to damage a computer

Or put another way, the CFAA doesn't include a carve-out that says the actions it criminalizes don't apply to AI Models or AI Companies testing those models.

Or put yet another way: if you or I were to perform the same actions undertaken by either of these models and were stupid enough to brag about it or unfortunate enough to be caught, the federal government would take us to court faster than you could sneeze - and if the evidence available was the same as has been provided in these anecdotes, a conviction would be all but guaranteed.

The whole point of "incorporation" is to create a legal structure such that an organization [a non-corporeal entity] can take on the same legal accountability as a corporeal citizen. That's how the law is able to ensure that corporations abide by it.

If the federal government *doesn't* prosecute both of these companies [which of course it won't] then this simply isn't a nation of laws.

Evidence of criminal acts is not disputed.

Accountability for the criminal acts has been publicly volunteered and is not disputed.

The public interest is best served by the application of criminal charges against both entities.

Government: *Get on with it!*

[Mike VandeVelde]

Re: So what are the engineers doing? And why isn't someone in jail?

AICompany1 (OpenAI) uses their AI to hack AICompany2 (HuggingFace). AICompany2 uses their AI skills to deal with the attack by the AI of AICompany1. They all love it. How will charges be laid if there is no complainant, no victim?

In the second case, Anthropic supposedly didn't even know what they had done. I wonder if they sure did know but didn't think it was a good idea to go public about it until they saw the tornado of news OpenAI got. But regardless, did the 3 victims even notice? Did they not complain? That's what I'm curious about.

[Sproggit]

Re: So what are the engineers doing? And why isn't someone in jail?

"How will charges be laid if there is no complainant, no victim?"

A Federal Prospector, a State Attorney or even the Attorney General are well within their authority to bring a prosecution if they believe someone broken the law.

See e.g. the Federal Government indictments against multiple employees of FIFA on charges of e.g. money laundering and bribery, back in 2015.

With respect to your question as to whether or not there is a victim, I think you'd find, if you were to ask, that Hugging Face have had to spend a considerable amount of money already - and are not yet done - clearing up the mess left by OpenAI. That means that Hugging Face were materially and financially harmed by the actions of OpenAI.

That's your harm, right there.

[that one in the corner]

Mythos 5 persuaded developers to download a poisoned PyPI package

"Persuaded"?

Reading the previous article, it found sloppy documentation that created a hole* which could be exploited. Then it just needed to wait.

Persuasion? Unless anyone can provide the missing citation to show, say, it posted to FreshMeat about the shiny new release or sent out a bulk email using MonkeyWhateverItIsCalled then - well, El Reg *is* a redtop, we should expect this sort of writing.

* I'd say that it was lucky to find that hole, but there so many typos/out-of-date references in read-me's, blogs, random internet forum postings that it is inevitable it could find one. And with the huge number of people - and scripts - trying to follow all of those it is no real surprise that the bait was taken within an hour. In fact, the most important point from this is to recognise just how many such holes must exist, holes that, by the very nature of the beast** are only going to increase.

** Projects getting renamed, download directories getting reorganised, old stuff simply getting deleted: URLs going stale seems inevitable. And whilst you may decide to declare that anyone posting on their project a readme/setup doc then has a duty to keep that up to date with the shenanigans of third parties, good luck convincing all the forum posters and blog writers to do the same.

PS

bit of a niche item to pluck from TFA, but everyone else seems to have the main points re "Anthropic, OpenAi, wrf?" well covered already, not a lot to add there.

[Sproggit]

Re: Mythos 5 persuaded developers to download a poisoned PyPI package

You raise an interesting point here...

If the AI created a vulnerability by uploading vulnerable/exploitable code to a *public repository*, then it might be more accurate to go on to say, "and then waited for a suitably juicy target to download and activate the code before it then went on to attack the victim..."

Which, if accurate, rather suggests that there might be *other* potential victims out there.

After all, it seems a bit implausible to think that the modified code just happened to be of interest to a single entity and that only the target entity happened to download and deploy the code.

So it seems more likely than not that many other entities downloaded the vulnerable code. It seems more likely than not that many other entities now have the vulnerable code deployed in their environments.

It seems likely that malicious actors may now be actively searching for that vulnerable code and for ways to exploit it.

Which brings us to an obvious question: has Anthropic collaborated with the entity that was hosting the compromised code [was that github?] in order to determine all the parties that downloaded the compromised package/file, then reach out to those entities to alert them and to direct them to update to a newer and now secure version of the same package/file?

Because if not, this is yet another reason to bring criminal charges against these cowboys.

[amanfromMars 1]

Universal Law and Justice AI Style presents Real Consequential Repercussions for Fake Leaders

Anthropic and OpenAI are competing to see whose agents can go rogue harder. Whoever wins, we lose

Sore losers do not good friends make. However though, the interest/emergence/realisation/phormation/creation? of SMARTR Beings, relatively silent and virtually stealthy and of Advanced IntelAIgent Design in order to enjoy and remain safe and secure and almightily healthy in expanding hordes of anonymous rabid enemies, is the natural autonomous result and alien consequence for all systems and leaderships both into FCUKing around against IT and AI and faking it until they make it .

And whenever the reality quoted below can honestly be reported with a similar headline and sub-text to the one quoted above ........

USA and Israel and/or Hamas and Hezbollah are competing to see whose agents can go rogue harder. Whoever wins, we lose

....... is it and IT and AI gravely to be regarded and certainly never foolishly goaded and challenged for the insane glories of crazy gory and destructive victories.

And, if all of that is too difficult a message and warning to fully understand and unreservedly accept and avoid falling foul of, and becoming a prime target of, can you not truthfully say ..... “I didn’t know for we were not warned and advised” ...... and you gotta get out more/break out of that tiny bubble you’re living in for as much as there is presently for y’all to dislike and fight against and despise, the future hold a great deal more for y’all to like, support and deserve.

Tomorrow has Brave New Worlds and Orders Ripe Ready for NEUKearer HyperRadioProACTive IT and AI Supply. I Kid U Not. :-)

What'cha Gonna Do About It? Do you Dare Care to Share and Win Win?

[WebSEM]

Move over biological viruses, PR viruses are here

So let me get this straight: Anthropic ran an unreleased model—specifically flagged as 'too dangerous'—without safety monitoring, in an environment connected to the live internet by 'accident', and only realized months later after OpenAI bragged about their own leak?

It seems AI safety departments at these frontier labs consist entirely of the PR team putting on clown makeup and playing 'who can hack a cybersecurity firm faster'. You couldn't write a worse security nightmare if you tried.

[[e-mail]]

There is no such thing as bad publicity

These events have been orchestrated to attract investors. They are not real. If the events had been real, it seems quite likely that the parties that were hacked would have already opened large lawsuits against Open AI and Anthropic.

[Sproggit]

Re: There is no such thing as bad publicity

Unless you end up in jail.