Re: Meanwhile
Methinks David doth protest too much.
I've read his blog post and also read the actual paper.
With all due respect, I believe Mr Gerard has fallen into a trap of "All AI Bad" and, by association, anyone associated with it is likewise bad, to some extent a trap of his own making.
The paper doesn't say what he claims it to say, to wit that it's basically marketing material from the AI companies rehashed.
It's not.
He may have read the paper but I don't think he actually got beyond the text itself and looked into the supporting material, of which there is quite a lot.
While I admittedly didn't read every one of the 445 papers in the metastudy but I did do a quick skim of a handful of them1 and if you actually look at the papers evaluated, the preponderance are from university research groups attempting to develop their own benchmarks and metrics, independent of the AI companies and not evaluating the AI companies own (allegedly) "thumb on the scale" performance benchmarks.
Now you may disagree with the results -- that's fine, that's what science is all about2 -- but unless your of the opinion that hundreds, if not thousands, of individual computer scientists are on the "take" from "Big AI," this isn't "marketing" material, it's honest research.
While I'm at it, I should direct you to yesterday's (11/7/2025) Pivot to AI posting, a guest article by computer scientist and cryptocurrency/AI skeptic Nick Weaver3 and especially the video/podcast interview4 where Weaver discusses the CS discipline of "machine learning" and its actual practical uses (as opposed to the self-fluffing hype of the AI bros).
_____________________
1 If I had one criticism of the paper, it's the fact they didn't provide clickable DOI links to them, which meant that I had to go through the extra step of feeding the titles to a search engine, but that's only a mild annoyance.
2 Preferably by writing your own paper and getting it accepted to a journal or a conference.
3 Pivot to AI: The futile future of the gigawatt datacenter — by Nicholas Weaver
4 YouTube: The futile future of the gigawatt datacenter (Interview with Nick Weaver).