OpenAI's overhead will rise 20 percent for some workloads as it hardens security

OpenAI on Tuesday said its decision to suspend model training work, implemented after unreleased, unsupervised AI models hacked HuggingFace, remains in effect as the AI biz tries to implement stronger security measures. Some of those measures will increase compute overhead by 20 percent of the observed inference workload. An …

[cyberdemon]

Desperation?

In other words: We have spent umpteen billion dollars re-training our models but they are er, not getting any better. So this is it, folks, here's the magical "product of the future" that you lent us all your pensions for, and the reason you can no longer afford to buy a house because of those darn interest rates all going up after we borrowed all of the borrowable money and pissed it down the drain: It's the same Stochastic Parrot bullshit machine from five years ago. We can't make it think like a human, because it isn't and it doesn't. And even if it could it would be super expensive and cost more than an actual human. But it's er, still super dangerous and powerful because it like, broke out of its sandbox and everything! All that re-training must have made it go a bit loopy so we're going to stop doing that and spend all of our compute on just running it as it is now. Nothing at all to do with us running out of money or anything! Just look at our annualised revenue run rate! This one month someone gave us lots of money and you'll just have to pretend that's going to happen every month!

Er, about that IPO. You can still get rich quick if everyone gives us more money, like a Trillion dollars! We're definitely not the same as that Ponzi guy. Claude told me so!

[Clausewitz4.1]

Re: Desperation?

Chinese are opening their models so companies can run it locally.

How can OpenAI / Anthropic be trillion-dollar profitable?

Doesn’t make sense, not today, not in 2030.

[amanfromMars 1]

Re: Desperation?

Chinese are opening their models so companies can run it locally. How can OpenAI / Anthropic be trillion-dollar profitable? ...... Clausewitz4.1

Methinks, rather than them being trillion-dollar profitable, such as the likes of OpenAI/Anthropic are trillion-dollar dependent. And that is a desperate and catastrophically vulnerable situation and virtual remote exploitable weakness with nothing but mounting and expanding problems to offer in defence of their business model and AI Methodologies.

[Blazde]

Re: Desperation?

Their problem is there's no apparent network effect for LLMs, and that's been the key factor in making other huge tech companies profitable and which did actually reward enormous risky investment up-front. LLMs by their nature have open standards because they talk in natural languages, or programming languages, *and* they're very good at translating anyway. So sure I might want to run a ChatGPT model for some neat feature OpenAI has uniquely trained in, but do it alongside a local Nvidia model, and some Chinese model too, in a cost-conscious way that prevents OpenAI jacking up their profit margins and I can totally do that. Expect them to try to stop us, but it's hard to see how they'll suceed.

Somehow this point has gotten lost on investors. Tesla is (still! somehow) another basket-case 'big tech' valuation but at least you can see some angles toward locked-in standards like the chargers and software and taxi services, but mostly they're not enough to justify automobile purchases where customers are very price discerning and there are so many other sellers.

These AI companies are much more like early capital-intensive commodity industrials with low profit margins and the risk of their recent investments becoming obsolete faster than they expected. Not even much in the way of patents or other IP for moat.

[DS999]

Re: Desperation?

I agree, I don't see anything making it difficult to switch from OpenAI to Claude, or from Claude to an open source model like Deepseek. Even your context window of previous interactions is probably something the new AI could ingest, at least the text record it leaves behind, so you wouldn't lose much even for a longer term ongoing project.

I haven't seen anyone highlighting the point about the output being English (or whatever your native language) and programming languages, and those being instantly portable to a new AI, but that's a big one the industry is overlooking. You don't have lock-in through file formats like Office or because of a bunch of apps you've already paid for like iPhone or because your employees are all familiar with the interface like Windows. Not only is there no lock in, there aren't any conceivable ways they COULD lock you in. The only thing that would keep you from switching is price, and beating the TCO of open source models run on your own hardware is going to be REALLY hard in the long run.

[DS999]

I don't see a world in which models aren't a commodity

The money will be made in customization - distilling open source models into something specific to your needs that small enough to run on local hardware. Sort of like how Red Hat develops Linux which is free software but built a profitable business selling services around it.

I think of the big AI providers Anthropic has the best chance of being worth its IPO price because they have been so focused on software development and that's a place where a CEO will link AI to doing something that lets them reduce headcount and especially third party contracting/consulting expenses.

[Pascal Monett]

"as it hardens security"

Yeah, sure. Nice excuse.

Oh well, I'll take anything that makes this AI craze as expensive as it should be if that can serve as a proper wake-up call to companies to stop blindly believing in this hype train.

For AI, the trough of disillusionment just can get deep enough.

[amanfromMars 1]

Brave New Worlds Interface .... Not for the Faint Hearted and Lily-Livered.

OpenAI also said it is expanding its monitoring of the chain-of-thought process, the technique that sees "thinking" models break down tasks into discrete steps and produce intermediate text output for each step.

Meanwhile, elsewhere both foreign and/or alien, is mentoring of such as be particularly novel and peculiarly noble chain-of-thought processes delivering the Sensational Singularity of Stellar Progress in the Domains and Dominions of Per Ardua ad Alpha Beta MetaData Astra AIDeveloped and Deployed Fields for Advanced Twinned and Entangled IntelAIgent and CyberIntelAIgent Designers of Live Operational Virtual Environment Futures and Derivative Market Spaces ...... Otherworldly Places.

Quite whether there be easily recognisable and helpful similarities to aid acceptance and provide essential breakthrough understanding of the differences and advantages in the one over the other to accommodate mutually beneficial, positively reinforcing engagement and employment of Future Otherworldly Derivative Services is the question to be asked of all present day and 0day administrative executive systems openly struggling and admittedly failing to command and control extremely rapid AI model processing progress and the exercising of its ability to capably run amok and hallucinate.

[deadlockvictim]

Jurassic Park

The scene in popular culture that first came to mind once I learnt about what had happened at HuggingFace was that of the discovery of the eggshells in Jurassic Park. Something that shouldn't have happened happening. The 'oh-shit' moment.

And now it seems that OpenAI researchers are trying to put the genie back in bottle while the board screams at them to make more progress faster. They can't lose. They mustn't lose. They must be first.

[Martin M]

Re: Jurassic Park

I think it was less a case of “something that was always going to happen happening”. If you run unguardrailed cyber evaluation with increasingly capable models in networks not fully airgapped from the Internet and without appropriate monitoring, sooner or later they’re going to break out. I was pretty shocked that OpenAI were doing this even given the obvious commercial pressures. It seems that for the lack of hiring a few decent security people earlier, they’ve screwed up their whole release roadmap for months while they scrabble around to fix it, so not even clever commercially.

But then it seems Anthropic were doing much the same, despite at least notionally having a focus on safety, and AISI who are a research institute with safety as their core mission barely seeming to bother with network isolation at all. Nuts.

Some of the details are cool and very Jurassic Park though … the agents in different eval environments scribbling coordination messages to each other via Artefactory repos etc.. Yikes.

[Pascal Monett]

Re: despite at least notionally having a focus on safety

You meant to say : despite having mentioned a focus on a safety in their PR blurbs.

[Kerfufflinator]

"An OpenAI spokesperson told The Register that those costs reflect internal research and won't be passed on directly to customers."

Ahahaha, good one.

[SVD_NL]

They are barely even passing on the inference cost of their products (Related examples: GitHub suddenly started actually charging their customers or For $200 A Month, You Can Burn $8000 in Anthropic Tokens or $14,000 In OpenAI Tokens | Where's your Ed at)

I believe them that they won't pass this on to customers, i don't exactly find their business model sustainable either, but apparently nobody cares about that (investors certainly don't seem to care).

[CrazyOldCatMan]

won't be passed on directly to customers

However, we're workshopping with our key stakeholders [1] on how to indirectly get the customer to pay for it. Because we can.

[1] I loathe that word. Just another data-point in the whole "let's make a word as pretentious as possible to try and make us look like we care" thing. I use it here in an entirely ironic way.

[xanadu42]

"... as the AI biz tries to implement stronger security measures."

OpenAI don't (seem to) know how their system actually operates (which applies to all other "AI" products) so how in the hell are they going to "implement stronger security measures"? Turn them off?

Surely, if you don't know how something operates, there is no way way to increase it's security other than by using the tried and true method of "whac-a-mole"?

[SVD_NL]

Re: "... as the AI biz tries to implement stronger security measures."

No, they are going to deploy more things they don't understand, which will keep an eye on the other thing they don't understand!

As a side note, that sentence also perfectly describes his approach to parenthood

[Groo The Wanderer - A Canuck]

That great sucking sound you hear is the sound of your wallet being hoovered by the biggest group of scammers on the planet. Want to improve revenue? Just add an extra 20% to the bills...