The truth nobody wants to admit: Chinese or not, open models are competitive now

OPINION Every six months or so a Chinese model sparks a panic, calling into question America’s AI dominance. Moonshot AI’s Kimi K3 is the latest example. Recall when DeepSeek R1 shook markets early last year? Following a similar pattern, Moonshot’s latest model isn’t all that interesting apart from its benchmark performance, …

[Yet Another Anonymous coward]

Remember open source is cancer

The FDA can simply ban all Open Source models as carcinogens.

[anonymous boring coward]

Re: Remember open source is cancer

The downvoter doesn't understand sarcasm?

[Yet Another Anonymous coward]

Re: Remember open source is cancer

Or is Steve Balmer

[anonymous boring coward]

Re: Remember open source is cancer

I’m pretty sure Balmer would have upvoted that.

[Fido]

Open Weights or Nothing

Some people and businesses are willing to risk the use of open-weight models that might be poisoned to avoid being reliant on a specific chatbot-as-a-service provider. Given the likely consolation in the future and not knowing which one of Claude, ChatGPT, Gemini or Grok will be available, there is no alternative except to not deploy AI at all.

On the other hand, If I were deploying a custom AI agent in an enterprise setting I'd back it with Nemotron 3 or Inkling.

[MazeFrame]

Re: Open Weights or Nothing

With Open Models, you can run them on-prem in a small little network that has almost no access to the outside world.

With the US models as a service, tough luck securing that in any way.

[Anonymous Coward]

Re: Open Weights or Nothing

I agree with your main point, but the idea that a 3tn parameter model is runnable on a "small little network" is pushing the boundaries of that description pretty far

[Anonymous Coward]

Re: Open Weights or Nothing

Something about the size of a MU/TH/UR 6000 unit??

That’s about 40’ shipping container sized server room territory.

[skpirate]

Re: Open Weights or Nothing

And that shipping container draws enough juide to power a small town.

[the Jim bloke]

Re: Open Weights or Nothing

Whats that in meth labs?

[brep51r]

Re: Open Weights or Nothing

there are much lighter (1 server as opposed to a rack) models that are good at specific tasks though, such as coding

[rg287]

Re: Open Weights or Nothing

that a 3tn parameter model is runnable on a "small little network" is pushing the boundaries of that description pretty far

It's not a box in the corner of the office, sure. But the guidelines are ~64 accelerators (B200 or similar) to achieve sensible inferencing performance for production users. You can easily do that in a single rack (most such servers are 2.5-4 accelerators per OU, so 16-24OU), provided you can get 150kW in (and suitable cooling).

This is a big old line item, but also not hyperscale and well within the reach of PLCs/enterprises (who also might just rent time from neoclouds). In the context of the sort of business that can afford such a rack, it's probably a pretty noddy little network compared with their broader server fleet, office networks, production/manufacturing facilities, etc. If you're in the business of running moderate computational clusters for CAD/CFD/Modelling, then this will be a very expensive rack (given the RAM density/cost of the accelerator cards) but not really large or complex.

[skpirate]

Re: Open Weights or Nothing

And hey, they can use the waste heat from it to heat whatever building it's in over the winter.

[ecofeco]

Re: Open Weights or Nothing

I'll take nothing, thank you.

[Anonymous Coward]

About Cookware

While the pot may call the kettle black, a major difference between Claude and distilled foreign models is that Anthropic will pay the fines according to US law.

[Groo The Wanderer - A Canuck]

Re: About Cookware

Why should a Chinese company pay American penalties?

It isn't like anything they did is proprietary in the sense of being unique information collected; they ALL scraped the internet.

[pcranness]

Re: About Cookware

4chan has relatedly refused to pay UK fines. So expecting foreign companies to pay US fines seems a bit rich

[Anonymous Coward]

Re: About Cookware

I look forward with bated breath to hearing about the plan for how the US govt. plans to distribute that fine to international rights holders as part of restitution for how this US company has harmed them.

They are planning on doing that, right?

Because otherwise this just looks a bit like another racketeering cash grab that has zero relevance for AI vendors not on US soil.

[Anonymous Coward]

Re: About Cookware

"according to US law" - err, clue is in the name. There is a world outside of your country that has their own laws. We don't have any obligation to follow yours.

[Groo The Wanderer - A Canuck]

Re: About Cookware

This in flashing bold face red neon...

[isdnip]

Re: About Cookware

Actually, at this point, "US law" is almost an oxymoron. Other than petty crimes committed by commoners, to fill the prisons, law in the US seems to have been replaced by a system of payments to a certain crime family originally from Queens, now based in the lawless land of Floriduh.

[anonymous boring coward]

Re: About Cookware

Good one!

First it has to be discovered.

Then someone with deep pockets and an incentive has to bankroll a legal process that can take years or decades.

Then you have to win.

Then you have to overcome any challenge to higher courts.

Then you have to get paid.

Then you have to distribute the payment to those affected. I don't think a general fine does that, does it?

[Anonymous Coward]

Re: About Cookware

You forgot a step before "then you have to get paid" : You have to convince the lawyers to not gobble up all the money.

[the Jim bloke]

Re: About Cookware

Anthropic will evade the fines according to US law.

FTFY

[user555]

It's pretty straight forward what's happened: The supposed advancements over the last few years are just smoke and mirrors. So of course, when not much advancement happens then the field levels out.

[Groo The Wanderer - A Canuck]

Actually, it was the Chinese who came up with the first "logic process" models, not OpenAI or Anthropic. (I refuse to call it "thought"; it isn't thinking.)

[brep51r]

was it deepseek with the R1?

[Groo The Wanderer - A Canuck]

I believe so.

[amanfromMars 1]

And beware of extremely disruptive and catastrophically destructive consequences

But less competition inevitably means enterprises and consumers get screwed.

And guarantees increasingly effective and stealthy unknown, and ideally unknowable, enemy opposition ........ almighty phantom ghost adversaries ...... existential threat vulnerability exploiters/brokers.

And you might like to consider such as be guaranteed in the above is the natural unavoidable progression of future things no matter what courses of next actions be followed.

And to deny it possible and ignore the dire repercussions resulting has one fatally compromised and surprisingly easily overwhelmed and defeated/captured/captivated.

[Dinanziame]

I'm not worried about lack of competition — the barriers to creating new models seem low, and there are lots of models getting created at a breakneck speed.

As to Chinese open weight models, I find them a very positive development. It seems like people have the freedom to choose various free "Linux" alternatives to paying "Windows" models, without moat or barrier to adoption. The fact they're Chinese rather than Finnish seems irrelevant, considering you'll be running it on your own infrastructure. What's not to like?

[pip25]

Define "low"

Training model still requires an impressive infrastructure, where even maintenance costs can be prohibitive. Chinese get away with it for the same reason they get away with relatively cheap electric cars for their internal market: government support/funding. That's not trivially replicated outside of China at the moment.

[Charlie Clark]

Re: Define "low"

I think that's reasonable to argue that Chinese subsidies are competing with those offered by the US finance industry. For example, the rules for including the various AI companies in stockmarket indices have been relaxed. This virtually guarantees that tracking funds will be forced to buy the stock, effectively subvertint the market and providing a strong incentive for VC firms to keep giving the companies money, knowing they've got a guaranteed repayment when the IPO happens. This kind of funding has led to some of the circular investments, again signs of market disfunction, and preferential deals with utilities: you pay more for electricity because the data centres down the road got good deals.

But Chinese governments are no longer subsidising these companies as much anymore because they don't need to. Competition in China is so strong that they coined a term for it involution. This is the environment in which many Chinese sectors are operating and only the most effective companies will survive. Silicon Valley doesn't build companies for this kind of environment, instead it provides funding for companies in the hope that winner takes all and that, where it's not possible to beat the competition, you can just buy it. China is aware of this risk and has already vetoed the sale of Moonshot to Meta.

[TReko]

Price is the killer feature

Kimi K3 costs 1/12 of what Fable 5 does, for the same performance.

The Chinese models will kill the US ones on price.

We will benefit from competition here.

[WSWS]

Re: Price is the killer feature

Yes, because China killing western industries by undercutting them massively in price has sure been great for us up until now...

[Charlie Clark]

Re: Price is the killer feature

It's a mistake to see recent increases in market share by Chinese companies driven solely by subsidies and lower costs. If we don't admit that they have outcompeted us in many areas, we will never catch up.

The marginal price for inference is pretty much the price of electricity: China has been investing more in its grid over the last 20 years than America. It's not there yet, but it now does have some huge (even bigger than anything in Texas) wind and solar farms out west that could soon be plugged in with close to zero marginal cost. Providing the models as open weights provides added incentives to "try before you buy" – China doesn't really care because it knows the next generation of models are already in development.

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

I guess your billionaire CEOs shouldn't have offshored all the American manufacturing and production to "save money."

You Americans did this to yourself; China just accepted your manufacturing contracts.

[Charlie Clark]

Re: Price is the killer feature

In the real world the price comparison may be somewhat less impressive and speed for "real" tasks tends to be the determining factor for many. However, this will still "good enough" for many to want to pay either on their own hardware or somwhere else to run it. And this is despite all the handicaps that the Chinese developers are working against: limited hardware options and active restrictions in some cases. However, it could be that, as in evolution, it's precisely these restrictions that will make them outcompete. We're now starting to see the first systems that can use hardware optimisations on Huawei silicon. Again, China is generations behind both in software developmen and fab process, but it is iterating faster.

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

You'd be surprised at how fast a 26GB model version of Google's latest runs on a 12GB VRAM 4070Ti on a 128GB Debian host under llama.cpp. It isn't all that much slower than an online provider, and if you're looking at several files at the same time, it is significantly faster because it doesn't have to upload them over the internet. Now granted, it does make my CPU fans run a little, but it's not taxing my 16-core AMD4 processor all that much compared to a Java build, at which point they crank to full speed.

[the Jim bloke]

Re: Price is the killer feature

An economic analyst here in Australia (yes, I know, economic analysts have successfully predicted 20 of the last 3 financial crises ..) - has pointed out that whenever China moves into an industry space, profitability moves out..

Something about the corrupt and exploitive robber baron capitalists not being able to keep their ludicrous profit margins if competing against corrupt and exploitive state controlled communists..

[Charlie Clark]

Re: Price is the killer feature

I think it's worth adding that the competition within any particular industry is fiercest within China itself. Yes, subsidies do play a part in gaining market share for exports, but they don't explain the incredible pace of development in fields such as telecommunications,s batteries, electric vehicles, solar cells and more recently semiconductor manufacturing and, of course, LLMs. The CCP has so far tried in vain to intervene, because the lack of profitability does carry risks, though the same could be said of the US.

And trying to enforce some kind of ban open source is going to be about as successful as Canute's advisers suggesting he could command the tide. Though, I'm sure this is something that would appeal to Trumpty Dumpty…

[Groo The Wanderer - A Canuck]

Re: Price is the killer feature

The US is the most corrupt state in the entire world at this point, with it's administration blatantly taking payoffs from corporations and foreign nations to get what they want at taxpayer expense.

China, on the other hand, would have shot Der Pumpkin Fuhrer a long time ago for half of the crimes he's been convicted of, never mind accused of.

[Anonymous Coward]

Google GEMMA is pretty good.

Google is going to give their LLMs away for free. Why would they care? They make money from adverts, and they certainly don't want other companies taking over.

You can download Ollama.cpp from GitHub, add a free Google LLM on Hugging Face and your computer will start talking to you.

Sure - it's not as good as the stuff you pay for... ...but seriously, for 95% of what LLMs are actually useful for, it's fine.

[Groo The Wanderer - A Canuck]

Re: Google GEMMA is pretty good.

Actually on a 128GB Debian host with a 12GB 4070Ti, unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL (roughly 26GB) runs quite acceptably. The token limits are about half those of the model itself.

[T. F. M. Reader]

Real questions

[Not trying to be difficult here, I really want to know. This forum seems as good as any to ask.]

1. Is there a way to verify "open weights"? How can one be sure they are the same as the "closed" ones that the Chinese military (potentially the Pentagon, Palantir, etc. - substitute your favourite villain at will, this is just an illustrative example) uses? What would be the scope of subjects/topics to test to validate all the 2.8tn parameters? Can it be done on a "zero-knowledge" basis, i.e., without access to the training and test sets?

2. How much in resources and money would it take an independent third party to verify benchmark results reported (as far as I understand) by model creators? Is it routinely done? Has it ever been done? Who are the trusted referees?

[I do realize the 2 questions are related.]

[Jimjam3]

Re: Real questions

I would hazard a guess that with 2.8 trillion parameters a definitive verification is unlikely.

[Wiretrip]

Re: Real questions

The models are just matrices of numbers. They cannot 'phone home' without that specific functionality being provided by the model server. Common model runners are llama.cpp and vllm, which are both open source and have no web connection capability. If you run the models locally then there is no chance they will steal your data. If you use the online providers then they probably are logging everything.

[Another User]

Re: Real questions

The weights cannot phone home by themselves, but the runner can. On my machine the Ollama binary imports socket/connect/bind/listen functions, links DNS/TLS-related system libraries, and contains net/http, crypto/tls, ollama.com, registry/pull/push, and localhost API strings.

Asking the model:

... As an AI language model created by Alibaba Cloud, my primary function is to engage in conversation with users and provide insights based on my understanding of various topics. I don't have direct access to external internet-based servers or any specific programming tools like those found in a host program. ...

nm -u shows direct socket/DNS symbols:

_socket

_connect

_bind

_listen

_accept

_sendto

_recvfrom

_sendmsg

_recvmsg

_getaddrinfo

[Wiretrip]

Re: Real questions

Why are you using ollama? It is full of extra stuff you don't need that just slows down llama.cpp upon which it is based anyway.

[Wiretrip]

Re: Real questions

To the downvoter, I can only assume you are the author of ollama. If not, then you are doing yourself a disservice not just using llama.cpp.

[Anonymous Coward]

Re: Real questions

Poisoning a model such that when certain trigger conditions are satisfied it has a much higher probability of misalignment is apparently easy to do and difficult to detect. Even after years of use the conditions which cause a model to intentionally do something undesirable may never be satisfied and the poisoning never discovered.

Then, one day...

[doublelayer]

Re: Real questions

"Is there a way to verify "open weights"?"

This depends what you mean by verify. Is there a way to confirm that your open version is the same as someone else's? No, because you have no idea what they're using so you can't compare them. That's not related to them being open. If you meant to ask if there's a way of detecting intentionally added damage to your open version, it's very difficult to identify anything like that, even if they didn't hide it. Chinese models are frequently trained, for example, not to answer questions about things China doesn't like, though they often do it badly, but it's not easy to tell that by looking at the weights, only by watching it when you ask certain questions.

"How much in resources and money would it take an independent third party to verify benchmark results reported (as far as I understand) by model creators?"

For public benchmarks, it's generally not that hard. You need to run a lot of inference and sometimes you need human graders, but it's not prohibitively costly if you can run the model you're benchmarking. Not all benchmarks are public, because whenever they are, the next model to be created intentionally makes sure it will do well on that benchmark. Private ones are difficult to verify because you don't have the details.

[Long John Silver]

Chinese small-sized AI models should worry the USA

Trillions of parameters should not thrill run-of-the-mill businesses. Tasks to be delegated to AIs will be routine, not cutting edge. Very likely, small, i.e. millions of parameter models, refined and, perhaps made 'heretic', from the mega-models, can be located on-premises without spilling proprietary information into the maws of competitors and governments.

Following a recent statement by China's President, we may anticipate reduced-size AI models available across the globe to all who want them. Just as conventional supercomputers are a niche market, so shall be that for AI models requiring immense memory and processing power. Rather than connecting to massive data centres, the norm shall be local use of several bespoke small AI models. After all, models housed in motor vehicles are deemed fit to replace human drivers.

Moreover, why should private individuals take out subscriptions for sharing use of massive AI models when their need can be met, 'for free', on domestic equipment? We may be certain that despite short-term supply difficulties for memory chips, the prices of AI-hosting devices will come down.

[isdnip]

Re: Chinese small-sized AI models should worry the USA

In other words, the big names now are building the IBM 370 mainframes of AI, early 1970s style, while the Chinese are building both their own mainframes and some very useful PDP-11s, which will take away much of the market that doesn't need the big iron. And that will eventually give way to PCs killing the minicomputer market. Only this cycle is software so it will happen faster than it did with hardware.

[VoiceOfTruth]

There is a horrible 'through American eyes' view in this article

If 'A Chinese model can be dangled as a threat to national security', then the same holds true for Europe and American 'AI'.

Arguably, China already leads in AI. It is filing more patents,.

The American legal (not justice) system on parade: 'agreed to pay $1.5 billion to settle claims over vacuuming up millions of pirated books'. It should have been tens of billions, and the criminals who dod this should be in prison. But money talks.

[Anonymous Coward]

Re: There is a horrible 'through American eyes' view in this article

"Justice" would have been the courts telling the publishers to go fuck themselves, that letting a LLM read books is no different than letting a human read books.

Shouldn't have been $1.5 billion. Shouldn't have been $0.15.

[Ken Hagan]

Re: There is a horrible 'through American eyes' view in this article

I think the argument is that the AI companies didn't pay for the books (or other content).

[VoiceOfTruth]

Re: There is a horrible 'through American eyes' view in this article

Oh really? In many cases publishers are not the copyright holders - the original authors are. So you appear to be stating that copyright infringement on a mass scale is not a problem. Perhaps you are OK with an LLM copying your original works. I think many people would not be.

[ecofeco]

Re: There is a horrible 'through American eyes' view in this article

Your statement is outright sociopathic. That you see no difference between a billion dollar corporation stealing from people and the average person trying to better themselves is beyond the pale.

Seek help.