Nvidia accused of cheating in big-data performance test by benchmark's umpires: Workloads 'tweaked' to beat rivals in TPCx-BB

Nvidia has been accused of cheating in a big-data performance benchmark, and thus unfairly coming out on top, by the very umpires of the test. At its GPU Technology Conference last year, Nvidia claimed that a cluster of its DGX A100 systems was 19.5x faster than the best performing system on the TPCx-BB benchmark, devised by …

[NeilPost]

Reissue the results

So publicly reissue the results with NVidia at the bottom ranked as “failed to run benchmarks”.

[Anonymous Coward]

@NeilPost Re: Reissue the results

Doesn't work that way...

Over the past 30 years there have been many who gamed the TPC benchmarks. Starting w Oracle and the TPC-C benchmark.

What the TPC.org will do is to remove the listing and ask NVidia to remove the marketing literature from their site(s)...

Not sure why they needed to game the system.

[_LC_]

Nvidia? Come on, they would never do such a thing!

Nvidia would never do such a thing. They would never get away with it - ever, pinky swear!

[Anonymous Coward]

Re: Nvidia? Come on, they would never do such a thing!

I guess this 15th bazillionith time... will be the last time. To be fair, they always state they've made "unbelievable advancements".

[Anonymous Coward]

Needs to be independently tested and results shown.

Perhaps if it's less, it might free up the Nvidia RTX stock if it's less than other number crunching boards.

[Anonymous Coward]

@AC ...

The TPC.org benchmarks are set and the company who submits their results has to have them audited by the TPC org before being posted.

A lot of the database companies stopped doing TPC benchmarks because to do an audited test they would need a hardware partner and they would need to spend a lot of money. It just wasn't worth it.

And no this isn't RTX related. NVidia has their own devices in their own boxes.

BTW, I do have an RTX3090 in a box to do some light work. 10K cuda cores is a lot for a desktop but their other stuff... much better if you can afford it. RTXs are cheap in comparison.

[Pascal Monett]

Nvidia has been accused of cheating

Many times.

Even when its hardware was the best, that still didn't keep it from "massaging" the numbers.

At this point, I think it is ingrained in its corporate culture.

[Anonymous Coward]

Re: At this point, I think it is ingrained in its corporate culture.

I've got this nasty feeling that it's ingrained in many, many "cultures". Starting from the top (gov), all the way to the bottom, where you put on this or that mask to appear more... intelligent, richer, prettier, etc, etc.

[gnasher729]

I remember many years ago, how a computer magazine couldn't reproduce some benchmark results at all. One was a text drawing benchmark, which would draw "The quick brown fox jumps over the lazy dog" again and again and again. The magazine's results were consistently three times slower.

It turned out they had a typo in their text, which would draw "the quick brown fox jumps over the lazy dog" with a lowercase t. And the graphics card driver had the string with the uppercase T built into the driver, and would use a pre-calculated bitmap for it, instead putting the individual letters together. So on this graphics card, the one string used in the benchmark would be drawn 3 times faster than any other text string.

[IGotOut]

Sue them.

Breach of copyright.

If they are using their name and logos etc without authorization, then they have every right to defend their name.

[Will Godfrey]

Nvidia cheating - not news

Nvidia giving honest information. Stop all the presses!