AI quota inflation is no token effort. It's baked in

OPINION Fans of the creative arts often find out where creators gather to talk among themselves, then sneak in to eavesdrop on what those masters of the art talk about. Golden insights, daring concepts, cutting-edge thinking? Not a bit. Gossip, if you're lucky. Travel miseries, if you're not. Mostly, they talk about money. It's …

[Headley_Grange]

Software engineers are going to become project managers of AI; forecasting token spend, monitoring token spend and producing powerpoints to explain why their token spend is greater than their forecast together with RAG metrics, token recovery plans, etc. Of course, they'll be pushed to use AI to draft the powerpoint, thereby increasing the token spend for no resulting value, but that's project management for you.

[Dan 55]

Sounds ghastly. Of course that means what's left of my career will be reduced to that and setting up agents.

[hittitezombie]

This is already here. It's depressing to work in a software house right now.

[Anonymous Coward]

Braindo

Where have I heard this before...

"AI the Task Mutilator had come to replace human ingenuity, virtually everywhere."

Tell me no one seriously thought all those compute units and GW would come for free?

[CatsRule]

Choose

Whether the civilisation we have built* over the last 10,000 years is for Humans or for Machines and their (temporary) Elites with all that implies...

The infantilisation of the population by these stochastic LLM/etc systems and their idiot overlords social experiments I would consider a "crime against humanity" - and they're just a simulacra of intelligence - can't wait for the real Artificial Super Intelligences (ASI) to appear

*I'm not saying were any good at it - but thats just, err - human?

[Bebu sa Ware]

if ((ntokens_left -= (strlen(prompt) + strlen(slop))) <= 0)…

"Ghastly. It all is – absolutely ghastly. Just don’t even talk about it."

[Throg]

Re: if ((ntokens_left -= (strlen(prompt) + strlen(slop))) <= 0)…

My very first thought was “you vibe coded that, right?”

[T. F. M. Reader]

Re: if ((ntokens_left -= (strlen(prompt) + strlen(slop))) <= 0)…

I assume it was vibe-coded, as it contains a bug. A review would notice that strlen(3) doesn't count the terminating '\0', but I bet LLM accounting does. Character-length tokens, though, probably fit right into the model. Or, at the very least, it wouldn't occur to Copilot to ask what the token length should be.

OK, OK, leaving...

[JimmyPage]

It's like maths reveals deep secrets in the universe

This is revealing deep fails in the understanding of intelligence,

(HINT: Intelligence is evolutions way of moving towards more efficient use of energy).

[b0llchit]

Re: It's like maths reveals deep secrets in the universe

We're not doing a great job, I guess. With the current motto of a few TeraWatts here and there extra we are on the course of:

Human intelligence is evolutions way to use as much power as possible to achieve less.

[Nik 2]

Re: It's like maths reveals deep secrets in the universe

I think this demonstrates that evolution has lost control of its creation, possibly to devastating effect.

What that has to do with a discussion about AI remains to be seen...

[Philo T Farnsworth]

Re: It's like maths reveals deep secrets in the universe

> (HINT: Intelligence is evolutions way of moving towards more efficient use of energy).

Or increasing entropy.

[Korev]

The token effort headline -->

[RojCowles]

Boldly leaping into the past?

Being an old I got my start working in an engineering design department in the early 80's before Unix workstations were much of a thing.

All the design calculations were run on a mainframe that we rented time on

You'd bang in your data, execute the particular program and the result would chunter out of the printer for your evaluation.

You also got one line at the end of the print job telling you how many seconds of CPU runtime you'd used and how much money your department was going to be charged for this time.

Being charged per million tokens for usage of these LLM based services almost feels like our tech bro overlords have managed to reinvent mainframe time sharing/renting about 50 years after it went out of fashion the last time.

[cd]

Re: Boldly leaping into the past?

Except the brilliant part is they charge for being inaccurate. After a while, who will know?

[Tron]

Time for a new computer revolution.

Well, that's GAFA for you. Nation states start wars, GAFA EOL and do subs. End users are just cannon fodder.

I would repeat my advice to avoid SaaS, Cloud, AI and subscriptions, as you would an STD. But in the case of AI, I really don't care. If you hop on that particular train, you have nobody to blame but yourself.

The only way of ticking all the boxes that people fret about is going to be through a new OS, designing out the endless flaws of the current ones, and a leap back to the future, to a time of applications on drives and servers. No SaaS, no Cloud and no AI. An OS that never requires updating for reasons of security is eminently doable, although you could add new bits every five years if you wanted to, without breaking backwards compatibility, should anyone actually invent anything new (nobody has for some time). Any silicon would do. Probably ARM. Compatibility with files and formats is all you need. Supply a development kit, and the usual suspects will offer works software in due course. Maybe 5 years to roll out if funded, by which time everyone will have been stung enough times a la Birmingham and be avoiding complex software packages, reverting to simpler tools.

[ecofeco]

It's all just extra steps

So far, LLMs have just re-invented what we already have.

But with extra steps. And lots of inaccuracies.

Far too many people are mistaking novelty for benefit. And the need for gatekeeping the novelty is as strong as ever.

[ind]

The article needs a smidge more editing, but this bit explains how ambiguity-by-design keeps economic control over users of ai.

"Basing costs on token consumption... makes as much sense [as] paying programmers per keystroke in and character out.... There is no concept of usable work actually done, no sense that inefficiencies are rewarded, and no easy way to relate the price paid to the actual cost of production."

Wargames reference was excellent. I also like The Simpsons' take on big, inanimate objects come to life, "Just don't look."

[tiggity]

Given how often "AI" produces incorrect / useless results to a particular prompt then often tokens are "wasted".

Fun for the people selling "AI" to the gullible - they receive income no matter how shoddy the output from their tool is.

At some point the predatory pricing to tempt people into "AI" will stop & charges will soar, I wonder if there may be some pushback then on only paying based on decent results.

[Sproggit]

Designed-In Robbery

From 1986-1998 I worked for on of the UK's many "Borough Councils" ... and when I joined the organisation in question proudly boasted not one but *two* "Honeywell-Bull mainframes". I think "mainframe" might have been a tad optimistic - this was a "Level 64" and an early "DPS 7" for anyone else of a certain vintage... and for those of the "Big Blue" persuasion, they were basically System 36 / AS/400 / iSeries machines in terms of their position in the range.

Those machines had an internal billing system baked in to the OS that was made up of 3 discrete elements - CPU time [and back then it was roughly £10 for a CPU-minute], storage space [and that was charged by the megabyte, and finally output - and that was charged on the number of printed lines ... all of which we could harvest from *basic* system accounting and all of which we could then cost based on our paper usage and hardware maintenance costs. [ We had to split out storage separate from CPU on account of certain departments insisting on using complete copies of their production data sets for testing...]

Anyway, I simply had my Shift Teams run one batch job per month... and it would collect that data from the machine's internal accounting files - all broken down by Cost Centre. I wrote some pretty basic COBOL to summarise the amounts, perform some calculations, then spit the results out in a file format that went directly in to the financial information system.

Simples.

If I could do that [checks notes... forty years ago...] I'm pretty sure that these AI companies can put some very simple metric capture subroutines in their code in a way that records actual CPU time and not "vibe coded tokens"...

The fact that they are *not* doing this tells me that the AI companies have intentionally created an abstraction between actual resource consumption and the cost being charged back to users.

Which tells me that they have done this in order to have the freedom to tweak their "charging model" in a way that allows them to continually adjust and adapt their billing in a way that will maximise their profits and not be reflective of either the resources users consume or the value of the service to those users.

In other words: they are price fixing.

Of course, in any other decade we might expect some form of regulatory interest... but given the recent changes to [decimation of] the federal government in the United States recently, I don't hold out much hope of there being anything that remotely resembles transparent, objective pricing for AI.

Which is a shame, really. I've used it quite extensively for the last 18 months or so and for some tasks it is frankly amazing [and getting better all the time]. But, powerful as it is, there is danger for any organization that invests in the technology only to be held hostage to the small handful of established players.

Which is pretty galling when you think that the effectiveness and efficiency of their product comes as a direct result of the millions of hours of testing and training that we have been providing to those companies - and then *paying them* for the privilege of teaching their models to get smarter.

Hmm...