Claude Code cache chaos creates quota complaints

Anthropic last month reduced the TTL (time to live) for the Claude Code prompt cache from one hour to five minutes for many requests, but said this should not increase costs despite users reporting faster depleting quotas. User Sean Swanson posted a bug report showing that Anthropic introduced a one-hour cache for Claude Code …

[IGotOut]

Rolls eyes....

Developers are stunned that hugely unprofitable business tries to claw a tiny fraction of their losses back by fucking over the people that don't pay enough to actually run the service.

[MyffyW]

Re: Rolls eyes....

I have one boss, a chap about my age but without my sassy flair for prose, who is convinced generative AI is the next reciprocating steam engine or world wide web.

I have serious doubts about the business model and very severe concerns about the likely future of civilisation. And therefore I'm spending my days running it down on the backwaters of the last technology wave. I now know how Alec Guinness felt playing Obi Wan. Except I've got better lip gloss.

[ErikOnTech]

Reminder: Nobody knows what any of this costs.

Right now Anthropic is dealing with balancing capacity constraints against usage, and doing so with an absolutely appalling lack of transparency: the kind that would and should get most companies sued into a smoking crater of oblivion.

So while the price remains the same, the value shifts wildly without warning or explanation. “Figure it out yourselves. Or, better yet, spend tokens on the problem we created for you.”

Even so, Anthropic is still burning through containerships full of cash every month. At some point they will need to charge what this stuff actually costs, which will raise prices considerably. And they'll have to ration tokens in order to provide reliable service, so as long as supply remains constrained get ready to bid those up.

We have virtually no insight into Anthropic's finances other than some cash flow estimates, but a good finger-to-the-wind guess is that prices will have to go up 5× to 10× in order to make the supply and demand curves meet in a place that provides decent ROI to Anthropic's investors.

Plan accordingly.

[David 132]

Re: Reminder: Nobody knows what any of this costs.

For "Anthropic", read "all AI providers"; I think they're all in the frantic "lose money on every customer but make it up in VOLUME!!!" stage of the hype cycle, or as they prefer to phrase it to investors, "customer acquisition".

[werdsmith]

Re: Reminder: Nobody knows what any of this costs.

More with Anthropic right now as they are the most successful of the coding AI providers and have become the go-to preferred platform.

But as the aim of these services is to grow customer base by providing subsidised product then as you say, inevitable without some innovation to improve the cost of supplying the service.

[Anonymous Coward]

Re: Reminder: Nobody knows what any of this costs.

However loose money on each customer scaled out here just means loose a fuck-ton of money.

There is no volume scaling efficiencies to be had here like producing boxes of chips, or watching shite on Netflix.

[snowpages]

Re: Reminder: Nobody knows what any of this costs.

LOSE not loose.

[JamesTGrant]

Re: Reminder: Nobody knows what any of this costs.

You lose an O or loose a belt.

[EdSaxby]

Re: Reminder: Nobody knows what any of this costs.

In Anthropic's case, you can see and smell enshittification coming. It usually seems to take longer, but I guess everything is faster with AI.

[TaabuTheCat]

At what price point does the shine wear off?

When it hallucinates for $20/mo, I expect no one cares a lot. But if it's still hallucinating at $2,000 (or more) a month, I get the feeling a lot of people are going to start questioning the spend. And like ErikOnTech says, no one knows what this really costs. $2,000/mo could turn out to be the starter plan.

[retiredFool]

Re: At what price point does the shine wear off?

I expect they(Anthropic) expect to charge what it costs for a human. 2000/mo isn't even the starter plan. 1000/day(8 hr) is the starter plan. If you want an "expert" level plan, expect expert prices, 1000/hr.

[Paul Herber]

Re: At what price point does the shine wear off?

Do you want fries with that?

[ErikOnTech]

Re: At what price point does the shine wear off?

I did some back-of-the envelope math in a substack article and came up with probably between 5× and 10× of current pricing, but it's a very complex problem.

Power generation is a huge constraint: the supply chains can't support anywhere near the number of gigawatts announced in anything resembling the time periods they were announced for. That whole industry was stretched thin in 2019, then the COVID lockdowns hit and threw sand in all the gears. So when the AI boom / bubble hit, a lot of manufacturing was already pretty close to the limit. Especially gas turbine generators, which involve far more exotic manufacturing than most people would imagine. The process for creating the fan blades is eerily similar to the process for creating silicon ingots for semiconductor manufacturing, only using alloys containing some of the rarest of rare metals. You can't just say, "Oh, we suddenly need 50% more of these (or even 15%)." The supply chain's response to that is “LOL. LMAO.” And that's just the first item on a long list of problems in that area.

[werdsmith]

Re: At what price point does the shine wear off?

People who pay 200 understand the limitations and work with it. The hallucination is a consequence of the process that makes it work.

[Someone Else]

Re: At what price point does the shine wear off?

And like ErikOnTech says, no one knows what this really costs. $2,000/mo could turn out to be the starter plan.

That "starter" plan would prove to be a non-starter.

These Tech-bro-B-school-dropout-Bill-Gates-wanabees need to understand the concept of pricing oneself out of the market. And if they don't learn beforehand, they will learn after the fact.

[kpgalligan]

Claude code going from 200,000 context to 1 million context, I think, was a huge mistake, Sure, people were complaining that they couldn't use 1 million context on the subscription, but those are people who don't have any experience using anywhere near that big of a context window. Models simply do not perform well once you get to that level. Maybe Opus is better than Gemini Pro, but it still is going to be struggling to be coherent. So the 400k limit is probably a reasonable cap and definitely should be used by anybody who's trying to use these tools. I cut it off after 250k-300k. That's outside of the context usage problem, which, of course, your usage is going to go way up, cache timeout specifics aside, if you're in a huge one million context window regularly. But the model itself and the agent itself are simply going to start malfunctioning. These are the kinds of things that people who don't really know how to use these tools simply do not understand or refuse to recognize even if they are made aware. It should "just work". People will say, "well, I haven't changed anything about how I use the tool". Sure, if how you use the tool is completely unaware of how it works, then you haven't changed how you've used it. But the tool itself has changed dramatically, and that's going to have an impact. So you really need to understand what your tools do. That's not to defend Anthropic or whatever. It's just, if you don't know what you're doing, this is a professional tool and you need to learn quick.

There are AI coding tools that attempt to keep users from doing things they maybe shouldn't. CC is more of a professional tool. It'll let you do more extreme things, but you need to know what the impact is. That said, I can see how the change to default thinking effort was missed by many, and that caused issues. But if you hang out on reddit, it's like Anthropic tried some huge rug pull. Anyway, it's been working fine for me.

[Anonymous Coward]

> I cut it off after 250k-300k.

Your comment about the usability of large context windows is spot on. It mirrors my experience with Chinese models. They're extremely clever until context starts filling up with disparate tasks. Always best to start a new session when switching tasks, and to make liberal user of /compress.

[Anonymous Coward]

Can someone ELI5 the significance of the cache?

I think I get it, but here's a concrete example of where I'm uncertain, Say I have a 750,000 llne XML configuration file (yes, I actually have one of these) and I want Claude to do analysis on the file, suggect changes, and generally use it as base knowledge for questions I want to ask. That's got to be 1M+ tokens, correct? So I upload the file and now I have five minutes to ask questions before it has to ingest the file again and burn up another 1M+ tokens?

[mIVQU#~(p,]

Re: Can someone ELI5 the significance of the cache?

Yes — you’re basically understanding it correctly, and the cache is a huge deal for exactly this kind of use case.

Think of the cache like temporary short-term memory for the exact prompt context.

When you first upload or include that enormous XML file (and yes, 750,000 lines could easily land in the hundreds of thousands to 1M+ tokens, depending on how verbose the XML is), the model has to process all of it once. That initial pass is the expensive part.

After that, for a short window (for example, around 5 minutes of inactivity, depending on the platform/model), the system can often reuse the already-processed prompt prefix from cache instead of re-reading and re-tokenizing the entire file every time.

So instead of paying for:

1,000,000 input tokens + your question

every single message

you may only pay for something closer to:

your new question + the changed part of the prompt

during the cache window.

That’s the significance.

A good ELI5 analogy:

Imagine handing someone a massive binder and asking questions about it.

• Without cache: every time you ask a new question, they must reread the entire binder from page 1

• With cache: they keep the binder open on their desk for a few minutes, so follow-up questions are fast and cheap

So yes, if you wait too long and the cache expires, the model may need to “read the binder” again, which means another large token hit.

The important nuance is that cache usually works best when the beginning of the prompt stays identical.

For example:

• same uploaded file

• same system/context instructions

• same conversation thread

Then only your follow-up questions change.

That makes repeated analysis workflows dramatically cheaper.

For a file that large, the best strategy is usually:

1. upload once

2. ask a burst of related questions within the cache window

3. keep follow-ups in the same thread

4. avoid restarting fresh chats unless needed

Something like:

summarize sections A–C

now suggest optimizations

now check for invalid dependencies

now generate a migration plan

That’s where cache shines.

One practical note though: even with caching, the model still has a maximum context window, so if the file truly exceeds the model’s context limit, it may not fit in one shot at all. In that case the system may chunk it, summarize it, or require working section-by-section.

So the short answer is:

Yes — the cache prevents repeatedly paying the full token cost for the same huge file, but only while the cached context remains alive and unchanged.

[T. F. M. Reader]

I don't pay Anthropic

One of the major reasons why (I have never had any particular urge to use Claude Code, but at some point I thought I might check out the shiny parts so admired by the multitudes) is that when I went to Anthropic's Pricing page I could not, for the life of me, figure out what it might cost me, how many tokens would a typical activity burn, what would be considered an API call, how many API calls would a typical prompt generate, what the limits (on tokens, API calls, etc.) would be, what would happen if I hit one or more of the limits (hard stop? graceful performance degradation? not-so-graceful rate increase?), and so on, and so forth.

It was quite a while ago (in AI time). Maybe it's better now. I don't think that I am the dumbest human being on this planet or that I was the only moron who couldn't figure something so trivial out. I simply decided I would not even try (an additional consideration was that someone who created such a convoluted pricing scheme was probably up to no good). Still happy with the decision.

[Anonymous Coward]

Re: I don't pay Anthropic

Fuck knows how any project or revenue work intending to use it can do any sort of development cost budgeting.

[abend0c4]

Re: I don't pay Anthropic

The same is true for more traditional cloud services - and in many cases they don't even allow you to put a strict upper bound on your payment.

We're about to find out what happens to businesses when bigger businesses control their infrastructure, determine their overhead costs, hold all their data and have the technology to clone their IT processes at will.

[AVR]

When you can't even see the size of the package before you buy

Ah, shrinkflation. We're getting plenty of it IRL right now, no surprise that the AI vendors have taken note and applied it to their product.

[MattieD]

This was always the end game

Anyone with a bit of savvy would have seen this as the outcome. All these AI companies have offered their products at well below cost to try and get people hooked on using them. Companies - and individuals - have embedded these tools into their workflows and business processes, usually sacking staff as a by-product.

Now, the AI companies - and more accurately the people funded them - want a return on their massive investment. Hence, the price goes up - be that by a direct increase or by 'shrinkflation' of what the customers get for their pennies.

All the companies that outsourced thinking and responsibility to the magic box of panaceas now find themselves in a tough position. They've no knowledgeable staff left and they're entirely reliant upon these AI tools. They're between the metaphorical rock and a hard place, and the only way out is to spend more and more money. Suddenly all those savings made from 'right-sizing staffing resources' are no-longer there.

I have no sympathy. You've made your bed, now lie in it.

[cd]

Claude, please meet your new pet, Algernon

[Zack Mollusc]

Hilarious!

Firstly, if "AI" was even half as useful as is being claimed, nobody would bitch about the price going up or the quality reducing slightly.

Secondly, everyone saying that the "AI" companies will need to increase fees to cover their costs. Cover their costs? The tech bros are trying to do the next NFT or crypto where you charge a fortune for providing bugger all. If your use of their platform costs them £10, expect to be paying thousands for the privilege.

[Graham Perrin]

For Those Of You Who Prefer Americanised Titles:

Claude Code Cache Chaos Creates Cuota Complaints