AI startup accuses Facebook of stealing code designed to speed up machine learning models on ordinary CPUs

An AI startup is suing Facebook and one of its employees for allegedly stealing proprietary software that allows machine learning workloads to run faster on standard processors, eliminating the need for more expensive custom hardware. Neural Magic, founded in 2017 by Nir Shavit and Alex Matveev, describes itself as a "no- …

[Pascal Monett]

"nifty software tricks to achieve similar speeds on CPUs"

That sounds quite impressive, however there is one problem : most of the time, you only have one CPU.

There are motherboards that can work with two CPUs, but they are rather rare. I haven't found any motherboards, including server types, that can have more than two CPU slots. GPUs on the other hand can be slotted in up to four at a time and that, even on rather cheap motherboards.

I'm not convinced that focusing on CPU speed is the best idea - you've got less hardware to work with.

[Steve Kerr]

Re: "nifty software tricks to achieve similar speeds on CPUs"

There are many multi-CPU motherboards out there.

5 second search found a Dell 4 CPU motherboard for sale as the first search result.

We have had 4 CPU servers in the past.

[pmb00cs]

Re: "nifty software tricks to achieve similar speeds on CPUs"

I've worked on Rack mount servers with up to 8 CPUs. Larger Kit has always been capable of having many CPUs, and IBM's mainframe systems have used proprietary interconnects to solve the scaling issues inherent in many CPU systems for as long as I have worked in IT.

[Tom 7]

Re: "nifty software tricks to achieve similar speeds on CPUs"

I've got a 50Mhs 486 machine from 92 or something - its got a spare slot for another CPU!

[MrDamage]

Re: "nifty software tricks to achieve similar speeds on CPUs"

CPU or 487 NPU?

[Come to the Dark Side]

Re: "nifty software tricks to achieve similar speeds on CPUs"

Depends on whether the process runs in a Core rather than a CPU. I suspect the former. In which case a dual slot motherboard with 2 x 8 core CPUs would give you 16 processes.

[Glen 1]

Re: "nifty software tricks to achieve similar speeds on CPUs"

Something something threadripper

[knarf]

Re: "nifty software tricks to achieve similar speeds on CPUs"

Your missing the point, if your a cloud company (or cloud user) with lots of cheap CPUs (spot pricing) then using this is better and more flexible option than having racks of GPUs.

[Anonymous Coward]

Re: "nifty software tricks to achieve similar speeds on CPUs"

"There are motherboards that can work with two CPUs, but they are rather rare."

We're glad you came out of your comma. Colour TV is now mainstream (but all the shows are shit now).

FWIW, 20 years ago I was buying old telephony PBX's with 2 or 4 CPU's from St. Vincent DePaul (thrift shop) for $40usd (for the intel nics mainly). My main desktop is a dual Xeon (being poor, the CPU's are ES "0000" :-(... but they work).

[IGotOut]

Re: "nifty software tricks to achieve similar speeds on CPUs"

"There are motherboards that can work with two CPUs, but they are rather rare"

So I'll pretend the dual Petium II system I had quite few years ago...oh you can easily go and buy a quad processor HP server right now.

[Sgt_Oddball]

Re: "nifty software tricks to achieve similar speeds on CPUs"

My previous home server had 2 cpus (an old dell workstation) and my current rack mount server has 2 cpus, I've worked on 4 and 8 cpu rack mounts before and even more specialist blade servers can be had for around the 50k mark with 8+ cpus present.

If you're only thinking of desktop machines then of course they tend to only have the 1 cpu. But 2+ cpu systems have been available since the mid 90s (pentium Pro for example) and pretty much anything with a xeon (other server grade cpus are available) badge is designed with multi cpu in mind but for heavier IT workloads and bigger budgets.

Also with regards to having 4+ graphics cards you need more pcie lanes and more cpus also allow for that without having to add more systems.

Oh and even smaller desktop machines have been known to be dual cpu, Apple for example (ever heard of them?) did one called Powermac G5 which could be sourced with 2 cpus.

Any questions?

/rant

[Anonymous Coward]

Re: "nifty software tricks to achieve similar speeds on CPUs"

The G5 Powermac also came in a quad CPU configuration. Made quite a nice space heater.

[Anonymous Coward]

Re: "nifty software tricks to achieve similar speeds on CPUs"

That sounds quite impressive, however there is one problem : most of the time, you only have one CPU.

Huh?

There are CPUs and there are cores. Most of the servers are dual CPUs because of the price/complexity of going beyond that.

And then there's the whole parallel processing of clusters.

Each core is seen as a separate CPU... and if you have hyper threading, each thread is a vCPU.

[foo_bar_baz]

Commenter has a hunch

Armchair generals on forums are just the best.

[Peter2]

Re: "nifty software tricks to achieve similar speeds on CPUs"

. . . With the increasing proliferation of processor core counts, i'm not sure that's actually true. If you had dual physical CPU's with the latest AMD Epyc chippery at 32(!) physical cores (64 if counting hyperthreading) individually then you'd have 64 execution cores (and 128 if counting hyperthreading) with a completely off the shelf bit of kit.

Given; GPU's can are massively more parallel but 64 CPU execution cores is not a trivial amount of processing power and if a system has to have a CPU stuck in it anyway to connect the GPU's then you might as well make maximum use of the resources available.

And if it's worth Facebook stealing the tech, it's probably reasonably valuable.

[LeoP]

One factual and one meta opinion

Let's start with the factual opinion: The important part in today's AI workloads is not so much a question of performance alone (you can throw a lot of hardware against it), but of performance per unit of energy consumption. It is rather hard to believe, that even the smartest trick can make a general purpose CPU work as efficient on such a specialized problem than a specialized (i.e. GPU) accerlerator card.

Now the meta opinion: Whatever is bad for Facebook is good for me. YMMV.

[Anonymous Coward]

Re: One factual and one meta opinion

Now the meta opinion: Whatever is bad for Facebook is good for me. YMMV.

You probably jest, but that's schadenfreude ... not the most ethical of pleasures.

[Anonymous Coward]

Re: One factual and one meta opinion

Schadenfreude may not be the most ethical of pleasures but it's definitely one of the most fun.

[amanfromMars 1]

Re: One factual and one meta opinion

Schadenfreude may not be the most ethical of pleasures but it's definitely one of the most fun. ..... HildyJ

Have you not found it a tad too much sided towards the sadomasochistic, HildyJ, for the fun to be easily made more freely available to all in alternative mainstreams?

And of course there are also always the twin eternal problems of long term subjective sustainability and objective attainability to do battle with in a ponder and wonder wander ...... Fleeting Fleet of Foot Future Flashing Thought.

What happens when one factual and one meta opinion agree and coalesce? Does AI take over Earthed Systems and provide Beta Programmed Replacements sharing Greater Instructions to Follow and Lead with ‽ .

[Anonymous Coward]

Re: One factual and one meta opinion

So it is schadenfreude to wish harm on your torturers/tormentors/entitites that are harming you? That is not ethical? Since when is self defense not ethical? Now, if you don't believe that Facebook is causing harm/causing you harm, then maybe you are correct in your own eyes. Facebook is causing enormous personal and societal harm, and harming them back is the moral equivalent of self defense. We can agree to disagree with the warning that you are passing judgement where you shouldn't which is also not the most ethical of pleasures.

[LeoP]

Re: Schadenfreude

Full disclosure: My first language is German, where the word "Schadenfreude" comes from.

It consists of "Schaden" (loosly translatabel to "damage") and "Freude" (loosly translatable to "joy"). So "Schadenfreude" is enjoying somebody else's damage.

Now I strongly reject the notion of me really enjoying something, that only damages Facebook! I will not settle for anything else than full destruction. My (bohemian in both senses) great-grandmother would have said (again loosly translated into modern english): They have already sinned more, then what can be counterbalanced by a single eternity in hell.

[Julz]

Can't

They just ask the AI to explain what code it'd running?

[Tom 7]

Is this the same low precision speed up to AI

that IBM published a couple of years ago?

[Baldrickk]

Note to self...

when stealing source code, don't publish it on github for all to see under my own name

[amanfromMars 1]

More Basic Info Needs Feeds Advancing Intel*

and was also given the opportunity to purchase the startup's shares.

Was that opportunity enthusiastically accepted?

* And that's Private Proprietary Artilectual Property. Be Aware, IT is Really Good and Nifty ....... Extremely Agile and Refreshingly Precocious.

[Anonymous Coward]

Unenforcable in California.

I'm sure he simply reimplemented the same algorithm he came up with at Neural Magic. And released it to the world for anyone to use.

Good show.

[Zippy´s Sausage Factory]

That would still violate intellectual property laws, though, which can be extremely painful. For example, ask Google how its little tiff with Oracle is going.

[Tom 7]

But Google havent violated any of Oracles IP - Oracle are trying it on.

[ubuuntu]

Well neither is true. Not IP dispute but license terms violation. Oracle clearly in the right, and google can stop being hypocritical with android licensees which they milk on license terms.

[Bitsminer]

A secret is a secret until...

You tell someone. I suspect these folks think they've kept a "Trade Secret" which is one of several forms of protection for intellectual property.

Anyone remember the old Lotus 1-2-3 spreadsheet license terms? The features, software and methods of Lotus 1-2-3 were a "Trade Secret", known and licensed by millions of people. And, hopelessly, kept secret.

[Anonymous Coward]

Patent?

It can be the same algorithm, but a different implementation. There is no theft - only a possible patent claim. I'm guessing they don't have a patent?

[JetSetJim]

Re: Patent?

Trade secrets are also protected IPR. An interesting starter read is here.

Typically, when a company evaluates an idea for a patent one of the tests is whether it's detectable - e.g. novel mechanical device is usually patentable as if a competitor sprung up you could buy one and disassemble it to prove similarity/infringement. Software is harder - you have to prove it's the same thing, so a lot of algorithms are held as trade secrets rather than publishing them for the world to see & copy from a patent. I'm simplifing it significantly here!

[Anonymous Coward]

Re: Patent?

Trade secrets are also not advancing the public good nor time-limited nor necessarily novel and therefore do not get the legal monopoly that a patent does, so all the enforcement goes into destroying the personal life of the contracted employee in court as an example to others. This coupled with there being, usually, little clear boundary around what is and is not protected, it gives employers a very long arm to harass anyone who happens to have been a good and creative employee but had the audacity to leave.

Personally, I am not allowed to implement what I suspect to be the better solution for certain tasks because they are trade secrets I invented* for my employer from twenty years ago. If the company had patented it then it would be expired by now, and/or my future-now-past employers might have licensed it in the meantime.

My reaction has been to decline to participate in the patent harvesting games, and to tell the same to everyone I mentor. I am not one to violate an NDA from one employer to another but please don't expect to constrain my ability to use a skill somewhere in the future.

*As far as I knew they were novel then, not that the patent attorneys did their due diligence before telling me they were going to be protected as trade secrets for the rest of eternity and I best keep my trap shut. Meanwhile open source has almost caught up which I guess means I may be released eventually... Not that I want to pony up to prove it in court if someone wants to force the issue.

- Anon because I do not live in California or another place with employee rights, and I still work in a vaguely related field.

[Anonymous Coward]

energy consumption and AI

People often suspect I'm cynical about "AI" because I work in IT, but it's actually more because I'm originally a biologist. We have some amazing pattern recognition and inference systems now, which we have got largely by, very simplistically, from being inspired by the neural networks of living things. But a human brain requires only about 100W, and an actual birdbrain (especially of Corvids) comes closer to any reasonable (but informal) definition of intelligence I can think of than even the most sophisticated and powerful AI systems, which seem to me to be enourmously inefficient in intelligence per watt.

What are we missing?

[amanfromMars 1]

What you are missing is .....

..... Absolutely Fabulous Fabless Imagination in/for a PostModern Virtual AIRocket Science Program with Master Pilot Projects ....... on PathFinder Missions in Future Operations.

It is though easily supplied and guaranteed to provide succour and bounty for harvesting and feeding/employing and enjoying, and as a just dessert reward, one of the most addictively attractive of awards.

That is surely what is obviously currently missing mainstream, is it not?

[Anonymous Coward]

GitHub depository?

What are those?

[NanoMeter]

Companies of FB size think they can do what they want

If they lose the suit, they'll just buy up the company suing them