Behold the Megatron: Microsoft and Nvidia build massive language processor

Nvidia and Microsoft announced their largest monolithic transformer language model to date, an AI model with a whopping 530 billion parameters they developed together, named the Megatron-Turing Natural Language Generation model. MT-NLG is more powerful than previous transformer-based systems trained by both companies, namely …

[Yet Another Anonymous coward]

And it will be used for.....

Predicting adverts

I see you are buying Nvidia 3090s and reading pages about games = I suggest porn

I see you are buying Grecian2000 and reading pages about Porsches = I suggest porn

I see you are buying Uranium and reading pages about time travel = I suggest porn

[Jedit]

"I suggest porn"

What does it suggest if I'm buying dildos and reading pages about porn?

[Mike007]

Re: "I suggest porn"

That one works. You get an ad for Grindr.

[Al fazed]

Re: "I suggest porn"

Windows 11

[Anonymous Coward]

Things are getting interesting in the world of pattern matching and data analysis.

But I still don't consider it "intelligent" by any stretch of the imagination.

It is often said that any sufficiently advanced technology will appear as magic to those who've never seen it before. I think we've got the same kind of blinders on when it comes to Artificial "Intelligence". We so desperately want something to be *right* and reliably *correct* in this world that we *need* these systems to be "intelligent" so *we* don't have to be...

[Anonymous Coward]

They're still not intelligent, all we've really done is improve the knowledge base system with a better inferencing engine and marketing

[Anonymous Coward]

We do it too

Labeling is important to companies pushing products. Artificial Intelligence sells more than Pattern Matching and Neural Networks sells more than Databases.

Then again, Information Technology sounds better than Data Processing which, truth be told, is, at its heart, what we do.

[Danny 5]

Nooooooooo

Did they have to call it Megatron? I mean come on, that's the same as calling a new supercomputer skynet.

[Guyver400]

Re: Nooooooooo

Especially given that his motto is "Peace through Tyranny"

[Tromos]

Cleansing toxic language

Do a bit more training. Scrape text from Facebook and Twitter and apply with negative weightings.

[Anonymous Coward]

Arthur C. Clarke., The Nine Billion Names of God (spoiler alert)

In a Tibetan lamasery, the monks seek to list all of the names of God. They believe the Universe was created for this purpose, and that once this naming is completed, God will bring the Universe to an end. Three centuries ago, the monks created an alphabet in which they calculated they could encode all the possible names of God, numbering about 9,000,000,000 ("nine billion") and each having no more than nine characters. Writing the names out by hand, as they had been doing, even after eliminating various nonsense combinations, would take another 15,000 years; the monks wish to use modern technology to finish this task more quickly.

They rent a computer capable of printing all the possible permutations, and hire two Westerners to install and program the machine. The computer operators are skeptical but play along. After three months, as the job nears completion, they fear that the monks will blame the computer (and, by extension, its operators) when nothing happens. The Westerners delay the operation of the computer so that it will complete its final print run just after their scheduled departure. After their successful departure on ponies, they pause on the mountain path on their way back to the airfield, where a plane is waiting to take them back to civilization. Under a clear night sky they estimate that it must be just about the time that the monks are pasting the final printed names into their holy books. Then they notice that "overhead, without any fuss, the stars were going out."

...

well, we'll get there, eventually! Or we'll extinct ourselves trying, yeah!

[Anonymous Coward]

Cleansing datasets

How hard can it be?

sed -i 's/fuck/fudge/ig' dataset

sed -i 's/bastard/ballard/ig' dataset

sed -i 's/bollock/bonker/ig' dataset

sed -i 's/shit/sugar/ig' dataset

sed -i ...etc...

[JDX]

Re: Cleansing datasets

I don't think the AI telling you to "go fudge yourself" is really that much better. And it's surely only going to confuse things when people actually use these replacement words... "Skynet tell me about the best fudge you ever had"

[bombastic bob]

Re: Cleansing datasets

I've often considered 'FEEL' to be "the F word"

So "go FEEL yourself" comes to mind...

[thosrtanner]

I have a natural language processor that doesn't require bucketloads of GPUs and concomitant wattage. It does take a little longer to train, but think of the power consumption benefits.

And it wrote this comment.

[Neil Barnes]

And I bet the manufacturers had more fun producing it, too...

[Cuddles]

Not that difficult

"Dealing with such large volumes of text means the dataset can’t be cleansed of toxic language."

No it doesn't. As with things like moderating social media, it just means the people responsible aren't willing to put in the resources required to do it.

[Tom 7]

Bigger is generally better when it comes to neural networks.

No it isn't, it really isn't.

[bombastic bob]

Re: Bigger is generally better when it comes to neural networks.

If they're ever to make it useful, they need to make it generate a set of AI parameters that can be applied to a much more modest hunk of hardware. Maybe the mega-AI-thingy with zillions of cores and parallelism is needed to establish those AI parameters, but once established, it should be possible to bake onto a single piece of silicon on a PCIe board or inside of a USB device... or just soldered onto a motherboard.

(I suppose it would be a kind of 'codec')

[Anonymous Coward]

Tell people you built a Megatron and they just stare at you blankly, but mention some hardware used by overgrown adolescents to watch anime and suddenly everyone's a data scientist. Can I continue uninterrupted?

[batfink]

GIGO

...regardless of the nice hardware spec.

[Kevin McMurtrie]

Units

How many PE teachers is that worth?

[Al fazed]

Bamboozle the behemoth language bulldozer

AI ? ML ?

Sure Microshits have heard of it. They built a GUI around using the START button to switch OFF your grey box, way back before I lost my memory.

They then moved on to enhance Windows performance by demanding Users experience the delights of mouse clicking twice instead of just once in order to STOP the machine - by using the START button.

By Windows 10, we all have to mouse click twice on the desk top screen before the display shows User a login text input field where password can be put in after clicking in the field. After accessing the desktop, User now needs two more clicks to close the delight which is the Edge browser that is getting in the way of doing any work. In fact Windows 10 is so fucking smart it lets me stop working altogether whilst it checks to ask the machine if it's ready for another dose of Windose Update............

Now were are told to upgrade to Windows 11 in order to solve the performance issues that came with Windows 10 update, in order to solve network vulnerabilities that were built into the PRINT spooler around WINDOWS 98.

I am also led to believe that the 8GB RAM fitted in side this box is necessary to make Windows 10 work with the speed of .................well, WINDOWS 10, or maybe a tad faster than WIN 3.1 used to be when running 4k of memory.

I suppose if we beleive everything than Microshitty meerkating spooky persons say when de-velopers have half finished knocking together the next version of their sheeze greater soft where ? The by using their version of AI and ML we are bound to arrive in the fabulous land of unicorn poo and rainbow shards, where hip idiots engage in synchronizzzing the unsyncable and only folks with white skin will be able to log in using parcial farcical recognition algoawayrythms

Anyone for more bollox, er I meant tennix..........

ALF