Chinese server builder Inspur trains monster text-generating neural network

Inspur has turned its hand to AI, and claims it has produced a text-and-code-generating machine-learning model superior to GPT-3 produced by OpenAI. And did so using significantly fewer GPUs. Inspur's model is called Yuan 1.0 and produces text in Chinese. The Chinese server maker says the model has 245.7 billion parameters ( …

[Anonymous Coward]

"That situation, he opined, means China will achieve military superiority within 15 to 20 years."

I'm really unsure why journos are so keen to propagate that kind of self-serving BS.

Oh wait, of course I'm sure why. Sensationalist clickbait.

[Anonymous Coward]

@a/c

"Oh wait, of course I'm sure why. Sensationalist clickbait."

Maybe you forgot to add the words, "I hope"?

[Anonymous Coward]

*LOL* You Americans won't lose your sense of un-entitled superiority until someone kicks your sorry posteriors down the street, and I do believe that is going to be happening in my life time. The American hegemony is falling apart, and once the bankruptcy for those trillions of debt comes home, the battle is over: America loses.

[BOFH in Training]

Well, look at the bright side -

If they are fighting in cyberspace, using AI to propagate false narratives, most humans will probably have a better chance of survival compared to a physical war, regardless with what weapons.

Of cos we may end up with the majority of the population not being able to distinguish reality with falsehoods.

Oh wait, thats already happening with all the QAnon / anti vaxers, etc.

[iced.lemonade]

that is more or less expected

the strength of any AI systems, ultimately, depends on the amount of data that is used to train it. the more data it churns, the better result it yields (i am no data scientist, but i feel if the amount of data increases beyond a certain amount, even if the data is biased, its effect will become insignificant - please correct me if i am wrong). in china every message sent, including voice and text messages, through any messaging software allowed in china is analyzed by the central government, for the statewide credit system, and given that the credit system is central to the political system in mainland china, naturally it is given unlimited amount of resources. i don't think any individual company can match the amount of data used to train china's AI - well, maybe if facebook, google and microsoft would pool their resource for a common AI initiative that may have a chance but i don't think that will happen anytime soon. i am not saying that the sharing of data between those 3 companies is a good thing, but technically, it takes a huge effort if the western side want to match, let alone outpace, AI development in china especially in the area of language processing.

[Anonymous Coward]

Re: that is more or less expected

"(i am no data scientist, but i feel if the amount of data increases beyond a certain amount, even if the data is biased, its effect will become insignificant - please correct me if i am wrong)."

Yes, and no.

The yes part. The data ends up in the parameters, that is why it matters how many parameters they can handle, 245.7 billion parameters in Inspur's model (that is double the number of neurons in a human brain). But, then think about it. The system uses 5 TB of samples to get there. Still, it's only able to fool observers 50% of the time. 5TB is a lot of language samples. There is no comparison with the amount of language a human encounters in her or his lifetime. Still, the computer model is not good enough to write a coherent text. That sounds like a very inefficient system. That also suggests that any improvement will take yet another order of magnitude more data and a similar increase in the number of parameters.

The no part. The problem for such natural language systems is that they have no good concept of meaning. That is why they do not generate reasonable texts.

On a fundamental level, it is not possible to extract "meaning" from only words. It is the old maxim of explaining to a person who never saw fire, what a flame is, or to explain the color "yellow" to a congenital blind person. But if you add interactions with the world to the model, you can start to add meaning to the words. Someone, somewhere will make this happen in AI controlled drones and robots, and then will be able to add meaning to the words.

So, there will be systems in the not so far future that will be able to add meaning to the language they input and use that to generate meaningful language in the output.

[iced.lemonade]

Re: that is more or less expected

thanks for your clarification

[Andrew Commons]

Re: that is more or less expected

"So, there will be systems in the not so far future that will be able to add meaning to the language they input and use that to generate meaningful language in the output."

Didn't Microsoft try something like that with Tay?

https://en.wikipedia.org/wiki/Tay_(bot)

[Pascal Monett]

Re: Someone, somewhere will make this happen

Yeah, well, I'll wait and see . . .

[osmarks]

Re: that is more or less expected

While sample efficiency is kind of a problem for many AI things, I don't agree with your explanation for why it happens or suggested fixes. Firstly, the comparison of parameters to human neurons is somewhat inaccurate, since actual human neurons are each connected to many others, and a parameter is kind of sort of analogous to the strength of one of those connections. Secondly, you can definitely develop some understanding of the world just from seeing text - natural language is a very rich source of information about all kinds of things, both because of direct explanations like on Wikipedia and the ability to make inferences from patterns of word use - for example, if you don't know what fire is, you can probably develop a rough understanding from e.g. references to it being warm, use of wood or other things to fuel it, and water stopping it from working. And having world models like this is incentivized, since knowing more general information allows more general and accurate text predictions than just directly memorizing which words are near each other often. I also think giving AIs robotic bodies is very wasteful, given that real-world interaction would act as a massive bottleneck, and that the idea of using other types of data (images, audio, etc) which can be obtained easily in bulk to augment textual training has already proven massively successful (see CLIP).

[osmarks]

Re: that is more or less expected

Nope. You can easily get all the training data you need by crawling the internet nowadays - a few orders of magnitude more than is needed for current models, even.

[Anonymous Coward]

Re: that is more or less expected

Witness recent faux pas by gaming companies who "crawled the internet" for their AI dungeon master...

[Mike 137]

"said to pass Turing test"

The problem with the Turing test is that the result depends not only on the capacities of the machine but on the observational capacities of the observer and the questions asked. It was never intended by Turing to be a 'test' of systems - it was a thought experiment. Rather like Moore's law it's been appropriated later for use where it doesn't really function.

That's not to say that this is not an interesting development. Work on natural language processing has been going on since the '80s and it's nice to see it coming near fruition.

[Andrew Commons]

Re: "Work on natural language processing"

"Work on natural language processing has been going on since the '80s and it's nice to see it coming near fruition."

I think you can trace it back to the 1950s. A small sample of papers from that period.

W. A. Clark and B. G. Farley, “Generalization of Pattern Recognition in a Self-Organizing System,” in Proceeedings of the 1955 Western Joint Computer Conference, Los Angeles, California, 1955, pp. 86–91.

Datamation, “Sarnoff foresees Voice-Controlled Systems,” Datamation, vol. 3, no. 7, p. 23, Oct. 1957.

D. L. Johnson, “The Role of the Digital Computer in Mechanical Translation of Languages,” in Proceedings of the May 6-8, 1958, Western Joint Computer Conference: Contrasts in Computers, Los Angeles, Calif., 1958, pp. 161–165.

W. W. Bledsoe and I. Browning, “Pattern Recognition and Reading by Machine,” in Papers Presented at the December 1-3, 1959, Eastern Joint IRE-AIEE-ACM Computer Conference, 1959, pp. 225–232. [Online]. Available: https://​doi.org​/​10.1145/​1460299.1460326

It was certainly well and truly on the radar and chess playing programs were being produced on the hardware of the day.

[Mike 137]

Re: "Work on natural language processing"

@Andrew Commons

Fair comment - I'm not that old, so I only started getting interested in it in the '80s.

[Andrew Commons]

Re: "Work on natural language processing"

@Mike 137

The 1950s and 1960s are really interesting periods where computing is concerned. WW2 saw the development of special purpose machines aimed at breaking ciphers. After that the concept of a generalized computer emerged and they were figuring out how to build them (they were one off and had individual NAMES), how to program them, and what they might be used for. AI/ML was of immediate interest right there beside A-bomb yields and other number crunching areas. The fact that business applications were different emerged quite rapidly in this period and influenced hardware development.

Now it's all kind of boring.

[Ken Hagan]

Re: "said to pass Turing test"

If you want to convince me that your system passes the Turing test, then I need to be the one picking the questions and I decide what subjects I can range over. Something that generates articles that can pass as written by humans isn't even close.

[Eclectic Man]

Re: "said to pass Turing test" - Anti-Turing Test

A genuine AI able to determine the difference between computer and AI generated poetry, fiction, etc. compared to human generated text would be very impressive, and truly scary.

[amanfromMars 1]

Re: "said to pass Turing test" - Anti-Turing Test

A genuine AI able to determine the difference between computer and AI generated poetry, fiction, etc. compared to human generated text would be very impressive, and truly scary. ..... Eclectic Man

And we should add, Eclectic Man, would be also quite totally alien and completely novel to humanised Earthlings.

Would that then pass Operational Field Systems Testing and Qualify for IDEntification in Programs and Projects and Pogroms as an Inscrutable Being and Enigmatic Body ..... if further information and enlightenment expanding upon the subject matter be needed to assist in root and branch explanation to provide greater clarification and deeper unhindered understanding of the PhenOmenA ????

Or would that be as Chinese or Russian or Korean or Yiddish or Gobbledegook in the West and, to be perfectly fair and honest, in many parts of the East too ?

[Anonymous Coward]

Re: "said to pass Turing test"

A question for Yuan 1.0's Turing test: "What's up with Xi and Winnie the Pooh?"

[Justthefacts]

Re: "said to pass Turing test"

The Turing test is not very *useful* in the real world. It shows only consistency, whereas truth is also usually required.

[Anonymous Coward]

Bring on the kaleidoscopes

I wonder if that 5 TB was all Party approved?

I dunno about military applications but it sure sounds like it's on the path to George Orwell's "versificators", machines used to construct media suitable for the entertainment of the proles.

[Anonymous Coward]

Re: Bring on the kaleidoscopes

I wonder if the Marvei Cinematic Universe is already using their own versificator to generate content?

[HuguesBalzac]

Re: Bring on the kaleidoscopes

They wouldn't need one as powerful as that, could probably get something suitable running on a Vic 20.

[Fr. Ted Crilly]

All you wanted to know...

-That situation, he opined, means China will achieve military superiority within 15 to 20 years.-

I think it would be extremely naive of us, Mr. President, to imagine that these new developments are going to cause any change in Soviet expansionist policy. I mean, we must be... increasingly on the alert to prevent them from taking over other mineshaft space, in order to breed more prodigiously than we do, thus, knocking us out in superior numbers when we emerge!

Mr. President, we must not allow... a mine shaft gap!"

[Uncle Ron]

My Toaster

I used to laugh when MS had all those ads on TV here about their "AI" and arranging photos and what-not. IMHO, MS has about as much AI as my toaster. I put in the bread, and it knows when to pop up. Same same.

[Draco]

FTFY

"China's government has made increased use of AI an economic priority, and places great store in the potential of the technology to improve services for surveillance of its citizens. "

[Anonymous Coward]

Re: FTFY

Well, one thing keeps bothering me about statements like that.

I've known and worked with a lot of Chinese nationals over the years who moved to Canada or the US, either on work visas or permanently.

They all looked forward to going home to see family. I met no one who avoided going back. I met no one who mentioned any onerous problems with growing up in China, and heard a lot of fond stories of childhood friends.

In short, the Chinese people I've known were quieter and more reserved than most, but really entirely normal, ordinary, everyday people. Different skills, different strengths, no two the same as anyone else.

Funny how every nation tries to paint pictures of "the enemy" as some "vile beast" ala Orwell's 2-minute hate...

[shraap]

Re: FTFY

Kind-of agree. Been living in HK for a number of years, and my observation on this kind of knee-jerk "China bad" is straightforward: just like you, I work with many mainland Chinese, and have found them just as normal as anyone else, with the same range of lovely people and jerks.

However, when we're looking at things politically, a distinction can (and needs to be) made between Chinese people and the Chinese Communist Party (CCP).

Chinese people good.

CCP bad. Very bad. And increasingly so. It's a very sorry view from here in HK, which is being trampled hard by CCP and their kowtowing lackeys here...

[FeepingCreature]

Re: FTFY

What? The criticism is about the chinese government, not the chinese populace. Are you sure you're replying to the right comment?

[Draco]

Re: FTFY

As someone who was born and raised (until we left) under Communism, I have no hate for anyone, nor do I see other people as my enemy.

Love of your homeland is strong in people - regardless of the government in power - but love of your homeland is not the same as love of the government.

I find the failure to distinguish between "the people" and "the government" very common among Westerners. I suspect, this occurs because Westerners have the luxury of political tribalism with its attendant hate of other political tribes. Or, in other words, people under Communism (or any Totalitarian system) don't have the luxury of political tribalism and all the "two minute hate" it rallies forth against its opponents.

Most people just want to get on with their lives without hating anyone or having anyone hate you. So you keep your head down and ignore the politics - except, it seems, in the luxurious West.

[Il'Geller]

These huge models can only be used for training: the best text in the collection discovered and applied to a given one. Nothing else... From a commercial point of view the models are useless for anything else.

[Anonymous Coward]

Well, if Microsoft invested in a dead, also-ran horse, you won't see me crying any crocodile tears for them. They've left developers out to hang while they churned their APIs and toolkits so many times over the years and forced me to suffer through it that I think they *deserve* a good blood-letting now and then. :)

[Primus Secundus Tertius]

Talking or listening?

It is not clear from this report whether that processor is intended to listen, talk, or do both. In the human world, someone who only talks is not welcome.

[yetanotheraoc]

Re: Talking or listening?

I don't think the processor cares whether you want to hear what it is saying.

[EarthDog]

Do they even know what the Turing Test is?

Here is the definition https://en.wikipedia.org/wiki/Turing_test

Excerpt from the wikipedia article * a three-person game called the "imitation game", in which an interrogator asks questions of a man and a woman in another room in order to determine the correct sex of the two players. *

So no, generating text is not the Turing test. The AI has to be able to hold it's own in a conversation which will then lead a human to infer the correct gender the machine is supposed to be imitating. Or the machine asking questions to humans to infer the humans genders.

[Anonymous Coward]

> "... humans who reviewed dialogues, news articles, poems, and couplets produced by the model could distinguish them from human-penned text "less than 50 per cent of the time."

That's not the definition of the Turing Test. The definition of the Turing test is being able to have a two-way conversation showing equivalent of honest natural human communication.

The doesn't include cute indirect answers and changing the topic which many systems seems to be good at.

[Jerry H. Appel]

The idea that China with the second largest economy in the world and largest population in the world will gain military superiority compared to the USA is a matter of sustained effort in house and via technology theft. Meanwhile, all the hubbub about hypersonic missiles is baloney to me. SLBMs and ICBMs get their faster and are still near impossible to shoot down. As to building a networked server system to perform AI with fewer GPUs, I will be impressed when said system shrinks down to the size of a volleyball and is self-propelled and able to find its own power.All this does is prove how remarkable the biological brains of Homo sapiens and let's not forget Cetaceans whose brains are even more of mystery.

[Anonymous Coward]

I realize the American mantra is to claim China "steals" all their technology, but in truth, your corporations GAVE them the technology so they could do their manufacturing for them, and they've built on that with some truly stellar R&D, much like any other nation.

If you think the Americans don't "steal" technology, take a look at what happens with a hostile takeover of a foreign company that OWNS the technology an American company wants... that is every bit as much "theft" as any other corporate espionage. The Americans just like to claim it is "legal" because they "paid" a pittance compared to what it was worth.

[Omnipresent]

Truth Destroyer

Sounds like the kind of thing that can take over Florida man's TRUTH social, should he ever get it off the ground.

You have a dictator over there driving his people into the ground waging war on the world from all fronts. He has economic power, military power, and cyber power. He even has space power. He is in love with power... he hungers for it. He is even building a race of generically altered super soldiers. You had better pay attention. He aims to be emperor of the known universe.

[amanfromMars 1]

Re: Truth Destroyer

The Donald would surely deny that characterisation of his powers and energy, Omnipresent, although be quite pleased that you noticed and recognised the progress of his efforts ‽ . On the strength of it though, he might venture to invite you to his Winter White House, Mar-a-Lago, to discuss or harangue you with what more can be more effortlessly done ...... for it is not as if the world is coming down with those able to share with him that which channels such energy into both positive covert and negative clandestine operations.

[TeeCee]

Not funny.

You know that "One China" policy?

Right. Imagine a globe. Now imagine it were all the land areas to be red and have "China" written on them.

Now you've got it...

[Anonymous Coward]

Re: Not funny.

To which I simply reply "America First?" Sound familiar?