The truth nobody wants to admit: Chinese or not, open models are competitive now

OPINION Every six months or so a Chinese model sparks a panic, calling into question America’s AI dominance. Moonshot AI’s Kimi K3 is the latest example. Recall when DeepSeek R1 shook markets early last year? Following a similar pattern, Moonshot’s latest model isn’t all that interesting apart from its benchmark performance, …

[Yet Another Anonymous coward]

Remember open source is cancer

The FDA can simply ban all Open Source models as carcinogens.

[anonymous boring coward]

Re: Remember open source is cancer

The downvoter doesn't understand sarcasm?

[Yet Another Anonymous coward]

Re: Remember open source is cancer

Or is Steve Balmer

[anonymous boring coward]

Re: Remember open source is cancer

I’m pretty sure Balmer would have upvoted that.

[Fido]

Open Weights or Nothing

Some people and businesses are willing to risk the use of open-weight models that might be poisoned to avoid being reliant on a specific chatbot-as-a-service provider. Given the likely consolation in the future and not knowing which one of Claude, ChatGPT, Gemini or Grok will be available, there is no alternative except to not deploy AI at all.

On the other hand, If I were deploying a custom AI agent in an enterprise setting I'd back it with Nemotron 3 or Inkling.

[MazeFrame]

Re: Open Weights or Nothing

With Open Models, you can run them on-prem in a small little network that has almost no access to the outside world.

With the US models as a service, tough luck securing that in any way.

[Anonymous Coward]

Re: Open Weights or Nothing

I agree with your main point, but the idea that a 3tn parameter model is runnable on a "small little network" is pushing the boundaries of that description pretty far

[Anonymous Coward]

Re: Open Weights or Nothing

Something about the size of a MU/TH/UR 6000 unit??

That’s about 40’ shipping container sized server room territory.

[skpirate]

Re: Open Weights or Nothing

And that shipping container draws enough juide to power a small town.

[the Jim bloke]

Re: Open Weights or Nothing

Whats that in meth labs?

[brep51r]

Re: Open Weights or Nothing

there are much lighter (1 server as opposed to a rack) models that are good at specific tasks though, such as coding

[rg287]

Re: Open Weights or Nothing

that a 3tn parameter model is runnable on a "small little network" is pushing the boundaries of that description pretty far

It's not a box in the corner of the office, sure. But the guidelines are ~64 accelerators (B200 or similar) to achieve sensible inferencing performance for production users. You can easily do that in a single rack (most such servers are 2.5-4 accelerators per OU, so 16-24OU), provided you can get 150kW in (and suitable cooling).

This is a big old line item, but also not hyperscale and well within the reach of PLCs/enterprises (who also might just rent time from neoclouds). In the context of the sort of business that can afford such a rack, it's probably a pretty noddy little network compared with their broader server fleet, office networks, production/manufacturing facilities, etc. If you're in the business of running moderate computational clusters for CAD/CFD/Modelling, then this will be a very expensive rack (given the RAM density/cost of the accelerator cards) but not really large or complex.

[skpirate]

Re: Open Weights or Nothing

And hey, they can use the waste heat from it to heat whatever building it's in over the winter.

[ecofeco]

Re: Open Weights or Nothing

I'll take nothing, thank you.

[Anonymous Coward]

About Cookware

While the pot may call the kettle black, a major difference between Claude and distilled foreign models is that Anthropic will pay the fines according to US law.

[Groo The Wanderer - A Canuck]

Re: About Cookware

Why should a Chinese company pay American penalties?

It isn't like anything they did is proprietary in the sense of being unique information collected; they ALL scraped the internet.

[pcranness]

Re: About Cookware

4chan has relatedly refused to pay UK fines. So expecting foreign companies to pay US fines seems a bit rich

[Anonymous Coward]

Re: About Cookware

I look forward with bated breath to hearing about the plan for how the US govt. plans to distribute that fine to international rights holders as part of restitution for how this US company has harmed them.

They are planning on doing that, right?

Because otherwise this just looks a bit like another racketeering cash grab that has zero relevance for AI vendors not on US soil.

[Anonymous Coward]

Re: About Cookware

"according to US law" - err, clue is in the name. There is a world outside of your country that has their own laws. We don't have any obligation to follow yours.

[Groo The Wanderer - A Canuck]

Re: About Cookware

This in flashing bold face red neon...

[isdnip]

Re: About Cookware

Actually, at this point, "US law" is almost an oxymoron. Other than petty crimes committed by commoners, to fill the prisons, law in the US seems to have been replaced by a system of payments to a certain crime family originally from Queens, now based in the lawless land of Floriduh.

[anonymous boring coward]

Re: About Cookware

Good one!

First it has to be discovered.

Then someone with deep pockets and an incentive has to bankroll a legal process that can take years or decades.

Then you have to win.

Then you have to overcome any challenge to higher courts.

Then you have to get paid.

Then you have to distribute the payment to those affected. I don't think a general fine does that, does it?

[Anonymous Coward]

Re: About Cookware

You forgot a step before "then you have to get paid" : You have to convince the lawyers to not gobble up all the money.

[the Jim bloke]

Re: About Cookware

Anthropic will evade the fines according to US law.

FTFY

[user555]

It's pretty straight forward what's happened: The supposed advancements over the last few years are just smoke and mirrors. So of course, when not much advancement happens then the field levels out.

[Groo The Wanderer - A Canuck]

Actually, it was the Chinese who came up with the first "logic process" models, not OpenAI or Anthropic. (I refuse to call it "thought"; it isn't thinking.)

[brep51r]

was it deepseek with the R1?

[Groo The Wanderer - A Canuck]

I believe so.

[amanfromMars 1]

And beware of extremely disruptive and catastrophically destructive consequences

But less competition inevitably means enterprises and consumers get screwed.

And guarantees increasingly effective and stealthy unknown, and ideally unknowable, enemy opposition ........ almighty phantom ghost adversaries ...... existential threat vulnerability exploiters/brokers.

And you might like to consider such as be guaranteed in the above is the natural unavoidable progression of future things no matter what courses of next actions be followed.

And to deny it possible and ignore the dire repercussions resulting has one fatally compromised and surprisingly easily overwhelmed and defeated/captured/captivated.

[Dinanziame]

I'm not worried about lack of competition — the barriers to creating new models seem low, and there are lots of models getting created at a breakneck speed.

As to Chinese open weight models, I find them a very positive development. It seems like people have the freedom to choose various free "Linux" alternatives to paying "Windows" models, without moat or barrier to adoption. The fact they're Chinese rather than Finnish seems irrelevant, considering you'll be running it on your own infrastructure. What's not to like?

[pip25]

Define "low"

Training model still requires an impressive infrastructure, where even maintenance costs can be prohibitive. Chinese get away with it for the same reason they get away with relatively cheap electric cars for their internal market: government support/funding. That's not trivially replicated outside of China at the moment.

[Charlie Clark]

Re: Define "low"

I think that's reasonable to argue that Chinese subsidies are competing with those offered by the US finance industry. For example, the rules for including the various AI companies in stockmarket indices have been relaxed. This virtually guarantees that tracking funds will be forced to buy the stock, effectively subvertint the market and providing a strong incentive for VC firms to keep giving the companies money, knowing they've got a guaranteed repayment when the IPO happens. This kind of funding has led to some of the circular investments, again signs of market disfunction, and preferential deals with utilities: you pay more for electricity because the data centres down the road got good deals.

But Chinese governments are no longer subsidising these companies as much anymore because they don't need to. Competition in China is so strong that they coined a term for it involution. This is the environment in which many Chinese sectors are operating and only the most effective companies will survive. Silicon Valley doesn't build companies for this kind of environment, instead it provides funding for companies in the hope that winner takes all and that, where it's not possible to beat the competition, you can just buy it. China is aware of this risk and has already vetoed the sale of Moonshot to Meta.

[TReko]

Price is the killer feature

Kimi K3 costs 1/12 of what Fable 5 does, for the same performance.

The Chinese models will kill the US ones on price.

We will benefit from competition here.

[WSWS]

Re: Price is the killer feature

Yes, because China killing western industries by undercutting them massively in price has sure been great for us up until now...

[Charlie Clark]

Re: Price is the killer feature

It's a mistake to see recent increases in market share by Chinese companies driven solely by subsidies and lower costs. If we don't admit that they have outcompeted us in many areas, we will never catch up.

The marginal price for inference is pretty much the price of electricity: China has been investing more in its grid over the last 20 years than America. It's not there yet, but it now does have some huge (even bigger than anything in Texas) wind and solar farms out west that could soon be plugged in with close to zero marginal cost. Providing the models as open weights provides added incentives to "try before you buy" – China doesn't really care because it knows the next generation of models are already in development.

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

I guess your billionaire CEOs shouldn't have offshored all the American manufacturing and production to "save money."

You Americans did this to yourself; China just accepted your manufacturing contracts.

[Charlie Clark]

Re: Price is the killer feature

In the real world the price comparison may be somewhat less impressive and speed for "real" tasks tends to be the determining factor for many. However, this will still "good enough" for many to want to pay either on their own hardware or somwhere else to run it. And this is despite all the handicaps that the Chinese developers are working against: limited hardware options and active restrictions in some cases. However, it could be that, as in evolution, it's precisely these restrictions that will make them outcompete. We're now starting to see the first systems that can use hardware optimisations on Huawei silicon. Again, China is generations behind both in software developmen and fab process, but it is iterating faster.

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

You'd be surprised at how fast a 26GB model version of Google's latest runs on a 12GB VRAM 4070Ti on a 128GB Debian host under llama.cpp. It isn't all that much slower than an online provider, and if you're looking at several files at the same time, it is significantly faster because it doesn't have to upload them over the internet. Now granted, it does make my CPU fans run a little, but it's not taxing my 16-core AMD4 processor all that much compared to a Java build, at which point they crank to full speed.

[the Jim bloke]

Re: Price is the killer feature

An economic analyst here in Australia (yes, I know, economic analysts have successfully predicted 20 of the last 3 financial crises ..) - has pointed out that whenever China moves into an industry space, profitability moves out..

Something about the corrupt and exploitive robber baron capitalists not being able to keep their ludicrous profit margins if competing against corrupt and exploitive state controlled communists..

[Charlie Clark]

Re: Price is the killer feature

I think it's worth adding that the competition within any particular industry is fiercest within China itself. Yes, subsidies do play a part in gaining market share for exports, but they don't explain the incredible pace of development in fields such as telecommunications,s batteries, electric vehicles, solar cells and more recently semiconductor manufacturing and, of course, LLMs. The CCP has so far tried in vain to intervene, because the lack of profitability does carry risks, though the same could be said of the US.

And trying to enforce some kind of ban open source is going to be about as successful as Canute's advisers suggesting he could command the tide. Though, I'm sure this is something that would appeal to Trumpty Dumpty…

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

The US is the most corrupt state in the entire world at this point, with it's administration blatantly taking payoffs from corporations and foreign nations to get what they want at taxpayer expense.

China, on the other hand, would have shot Der Pumpkin Fuhrer a long time ago for half of the crimes he's been convicted of, never mind accused of.

[Anonymous Coward]

Google GEMMA is pretty good.

Google is going to give their LLMs away for free. Why would they care? They make money from adverts, and they certainly don't want other companies taking over.

You can download Ollama.cpp from GitHub, add a free Google LLM on Hugging Face and your computer will start talking to you.

Sure - it's not as good as the stuff you pay for... ...but seriously, for 95% of what LLMs are actually useful for, it's fine.

[Groo The Wanderer - A Canuck]

Re: Google GEMMA is pretty good.

Actually on a 128GB Debian host with a 12GB 4070Ti, unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL (roughly 26GB) runs quite acceptably. The token limits are about half those of the model itself.

[T. F. M. Reader]

Real questions

[Not trying to be difficult here, I really want to know. This forum seems as good as any to ask.]

1. Is there a way to verify "open weights"? How can one be sure they are the same as the "closed" ones that the Chinese military (potentially the Pentagon, Palantir, etc. - substitute your favourite villain at will, this is just an illustrative example) uses? What would be the scope of subjects/topics to test to validate all the 2.8tn parameters? Can it be done on a "zero-knowledge" basis, i.e., without access to the training and test sets?

2. How much in resources and money would it take an independent third party to verify benchmark results reported (as far as I understand) by model creators? Is it routinely done? Has it ever been done? Who are the trusted referees?

[I do realize the 2 questions are related.]

[Jimjam3]

Re: Real questions

I would hazard a guess that with 2.8 trillion parameters a definitive verification is unlikely.

[Wiretrip]

Re: Real questions

The models are just matrices of numbers. They cannot 'phone home' without that specific functionality being provided by the model server. Common model runners are llama.cpp and vllm, which are both open source and have no web connection capability. If you run the models locally then there is no chance they will steal your data. If you use the online providers then they probably are logging everything.

[Another User]

Re: Real questions

The weights cannot phone home by themselves, but the runner can. On my machine the Ollama binary imports socket/connect/bind/listen functions, links DNS/TLS-related system libraries, and contains net/http, crypto/tls, ollama.com, registry/pull/push, and localhost API strings.

Asking the model:

... As an AI language model created by Alibaba Cloud, my primary function is to engage in conversation with users and provide insights based on my understanding of various topics. I don't have direct access to external internet-based servers or any specific programming tools like those found in a host program. ...

nm -u shows direct socket/DNS symbols:

_socket

_connect

_bind

_listen

_accept

_sendto

_recvfrom

_sendmsg

_recvmsg

_getaddrinfo

[Wiretrip]

Re: Real questions

Why are you using ollama? It is full of extra stuff you don't need that just slows down llama.cpp upon which it is based anyway.

[Wiretrip]

Re: Real questions

To the downvoter, I can only assume you are the author of ollama. If not, then you are doing yourself a disservice not just using llama.cpp.