Hugging Face rebuilt a third of its infrastructure after OpenAI agents ran amok

Hugging Face rebuilt around a third of its infrastructure from clean images as part of a sizable cleanup effort following the OpenAI security mishap earlier this month. The revelation is among several additional details disclosed in a postmortem published Monday by the Cloud Security Alliance (CSA), with input from Hugging Face …

[Sproggit]

Cover the Cost

The very least that OpenAI should do - right now - is publicly state that they will cover the full cost of all work that Hugging Face have to do to tidy up this marketing-ploy-gone-wrong.

Not because I'm demanding some sort of altruistic behavior from them.

But because, if they don't, other companies are going to be rightly cautious of deploying OpenAI models in security capacities, out of concern that the model will simply "go rogue" and start to perform *Criminal Acts* against other organizations.

https://www.justice.gov/jm/jm-9-48000-computer-fraud

https://www.legislation.gov.uk/ukpga/1990/18/contents

etc.

The marketing ploy for OpenAI would raise some concerns: "Deploy our Model for your cybersecurity needs - and it will break the law, so you don't need to!"

Personally, I would like to see the DOJ open a case against OpenAI, under the Federal Statute, then take it all the way to a prosecution [if the facts merit]. Not because I think OpenAI need to be singled out for special or particularly harsh punishment. I'd be happy with a token fine.

I think this is needed as a "warning shot" across the bows of all AI model developers and a reminder that if you use technology to break the law, it's still you that's breaking the law. The companies clearly lack either a sufficiently strong moral compass and/or basic common sense.

We should look upon this incident as an opportunity to prevent future repeats - and a big part of that is issuing an unmistakably clear warning.

[Anonymous Coward]

Where are the arrests?

Huggingface should just file a police report with the FBI. They can then lift the OpenAI management team off their beds in the early morning.

I don't understand why Sam Altman is still walking around instead of in custody awaiting trial. Thousands of people have been jailed for breaking into other people's computer systems but here they are getting away with an 'Oops' and a marketing opportunity.

[Cav]

Re: Where are the arrests?

You do understand the difference between intentional and accidental?

They might be prosecuted for negligent harm but can't be prosecuted for a cyber attack when that wasn't their intent. Actus reus is obvious but you'd have to prove mens rea in this case. You'd have to prove that OpenAI could reasonably be expected to know that their model would escape but they were negligent in allowing it to run anyway.

[kurios]

Re: Where are the arrests?

The counterargument is that doing what OpenAI did is an inherently dangerous activity for which lack of intent is irrelevant. Most legal codes and interpretive jurisprudence recognize this concept.

[Pulled Tea]

Re: Where are the arrests?

Robert Tappan Morris was tried and convicted for accidental breach into computer systems because of a program he made.

Morris was tried and convicted of violating United States Code Title 18 (18 U.S.C. § 1030), the Computer Fraud and Abuse Act,^[14] in United States v. Morris. After appeals, he was sentenced to three years' probation, 400 hours of community service, and a fine of US$10,050 (equivalent to $23,800 in 2025) plus the costs of his supervision.^[15] The total fine ran to US$13,326 (equivalent to $31,500 in 2025), which included a $10,000 fine, $50 special assessment, and $3,276 cost of probation oversight.^[16]

Legislation already exists. All that's left is enforcement.

[Anonymous Coward]

Re: All that's left is enforcement

They probably already donated to the White House ballroom fund. So they're good.

[just4this]

Re: Where are the arrests?

It seems they may still be compiling a list of victims.

OpenAI says its rogue AI tried to hack other companies.

https://www.bbc.co.uk/news/articles/c2el319vzr3o

[Irongut]

I hope Hugging Face are suing OpenAI for damages and loss of business caused by this hack.

Also when will the DOJ arrest Altman for misuse of computers? A system developed and ran by his company hacked another company. There should be jail time.

[Cav]

"when will the DOJ arrest Altman for misuse of computers"

never, and nor should they.

The US CFAA (U.S. Computer Fraud and Abuse Act) requires intent. There is no evidence that this was a deliberate act.

Under the doctrine of vicarious liability, you can sue a company for the actions of its employees but, in this case, OpenAI employees did not intend or order the attack.

Hacking law does not include provisions for accidental hacking. Legally there is no such thing.

It's why major tech corporations are not liable if their products lead to hacked or damaged computer systems.

The only option you have is to prove negligence under civil tort. But then you have to prove that OpenAI could reasonably be expected to know that their model would escape and attack other cyber infrastructure, but went ahead anyway.

And before the down votes: what we think should happen to OpenAI is irrelevant. The law is the law and people\corporations can only be prosecuted according to the law.

[FIA]

The only option you have is to prove negligence under civil tort. But then you have to prove that OpenAI could reasonably be expected to know that their model would escape and attack other cyber infrastructure, but went ahead anyway.

From the article:

In pursuit of the test answers, the models reportedly chained vulns in the dataset processing pipeline to achieve remote code execution on a processing worker, before hoovering up cloud and cluster credentials over the course of four days.

CSA's report, authored by CISOs, noted that two days were spent on reconnaissance, followed by a day of quiet, and the final fourth day involved "intense activity."

Wouldn't you have a reasonable argument to negligence though?

The model was supposed to be sand boxed.IIRC it abused a bug in a package cache. A curated list of acceptable package download locations isn't hard to generate, network monitoring tools are not uncommon. A 'highly capable' model with guardrails disabled should've triggered at least one 'Hmm, why is all this weird traffic happening' alarm. It's not an unrealistic scenario to expect this when planning this kind of testing, especially given the capabilities of the model being tested.

I think you could argue they should have reasonable expectation that their model, removed of all guardrails, could do something like this and put appropriate monitoring in place.

[Pulled Tea]

The US CFAA (U.S. Computer Fraud and Abuse Act) requires intent.

Nah.

Is it a slam-dunk? Nope. But there's precedent for it.

[Roland6]

"Intent" includes deliberately turning a blind eye. Ie. OpenAI did not demonstrate and have not demonstrated they were and are reasonably overseeing and controlling their systems...

[Anonymous Coward]

Non-Technical Suggestion................

..............air gap!

[l8gravely]

The birth of "Ralph the Wise and Powerful", a reference to Daniel Keys Moran's "Continuing Time" series...

[Taliesinawen]

Ralph the Wise and Powerful

> The birth of "Ralph the Wise and Powerful", a reference to Daniel Keys Moran's "Continuing Time" series...

“Hi there. This is Eddie, your shipboard computer, and I'm feeling just great, guys, and I know I'm just going to get a bundle of kicks out of any program you care to run through me.”

[CrazyOldCatMan]

Re: Ralph the Wise and Powerful

“Hi there. This is Eddie, your shipboard computer, and I'm feeling just great

[Reaches for fire axe]

I won't be a moment guys. Just going to visit the computer room for some manual reprogramming..

[Tron]

This is AI on AI violence.

So I'm struggling to give a toss.

But it seems to come down to simply having inadequate security. It doesn't matter whether it is a script kiddie, a malware gang, an agent or a bot. Your system should be secure. The more of a target you are, the better your security needs to be.

Oh, and why do they need to disable 'cheating'? Isn't that a design flaw? Not 'cheating' should be the default.

[brainwrong]

Cheating

"although it was not told that cheating was disallowed in its underspecified prompt."

Do these models have any concept of cheating, permissions, ownership, or rules? I see no evidence of this.

[Pirate Peter]

Re: Cheating

they have no ideas like "concepts" AI models are just massive probability databases, there is no intelligence, its works purely on the face B follows A etc based on the data it was trained on

if a probability of the answer you need is not covered by its training data it will just make shit up / lie (hallucinate) often in a convincing manner

[IGotOut]

Still not buying it...

...until I see real world consequences for OpenAI, I'll just call it more bullshit hype.

[Pulled Tea]

This was a crime.

We already have precedence for this. Hugging Face don't need to do anything — the United States government can take action, since similar cases ended with felony convictions.

Whether they intend to do so, however, remains to be seen, especially in light of the state of legal enforcement in America right now.

[CrazyOldCatMan]

Re: This was a crime.

Well - OpenAI isn't a minion/slave/favoured pet of the giant orange sleazeball so he might unleash his DOJ puppets on them, just to do them some damage.