OpenNvidia could become the AI generation's WinTel The OpenAI and Nvidia $100 billion partnership sure sounds impressive. $100 billion isn't chicken feed, even as more and more tech companies cross the trillion-dollar mark. But what does it really mean? As two of my Register colleagues noted, "The announcement has enough wiggle room to drive an AI-powered self-driving semi …
Wintel brought standization. USB, UEFI, etc. Which was and is good. Also for running Linux on your X86.
Look at the mess in the ARM world, both mobile and laptop (Snapdragon, Apple Silicon). Everybody is doing something and something different. Ubuntu on Snapdragon is still not fully working.
In my last job before retirement, the software I was product manager for ran on Intel (mainly Linux, but also Windows). I would get requests from the salesforce about once a month for us to port it to ARM because they had a prospect. (This was a time when ARM was starting to crop up in reasonable numbers in the big cloud providers).
And I would always reply "I'd be happy to do an ARM port, but which one?". No two requests were for the same implementation.
And I think this reflects the fact that a lot of people in the industry are too young to know a world before Wintel (or too sheltered to know the one beyond it), and do not understand that Wintel is not even just a chip and an OS (or several OSes), it is an entire system architecture.
Apple uses UEFI (and doesn't lock the bootloader) and were the first to adopt USB.
The difficulty of Linux on ARM is down to drivers. Both Apple and Qualcomm have proprietary GPUs. Apple has no interest in running Linux on Macs, so while they don't stand in the way they aren't going to devote resources to helping. Not sure if Qualcomm is helping but if so I guess they aren't helping enough.
You act as though Nvidia/AMD/Intel GPUs are based on open standards. Intel at least I will give credit for writes Linux drivers for its GPUs and pushes them to the kernel maintainers so they are ahead of Nvidia & AMD in that regard who only distribute binary blobs (or you can use lower performing reverse engineered open source drivers if your GPU isn't too new to be supported yet)
There are other issues like chipsets where reverse engineering stuff like controlling all the cores for optimal power saving is not easy so things don't work as well as they do in the x86 world where those (again, proprietary!) chipsets from Intel and AMD get open source drivers courtesy of the manufacturer.
None of this has anything to do with x86 vs ARM, but depends rather on the self interested decisions of the companies involved.
There's a crucial difference, though. Nobody (who had eyes to see) ever had any doubt that PCs and the Internet would be a history-shaking global revolution. There were hundreds of avenues of useful development that were very clear and well laid-out, already solved in theory, and only needed some work to become reality. The reason we had the dot-com bust was because there was uncertainty about the shape and speed of that.
There is quite a lot of doubt that LLMs are going to be like that. ROI is still failing to manifest, except for some specialized cases. Avenues of development are highly speculative, and rely on wholly unproven assumptions, for example that problems with current models can be solved with bigger models, or better training sets. But there is, in fact, a reasonable possibility that LLMs just won't get much better than what we have now, and turn out to be mostly useless in practice.
If LLMs fizzle, OpenAI will die, and Nvidia will not feel very well either. I would not stake my pension on that.
"The reason we had the dot-com bust was because there was uncertainty about the shape and speed of that."
Indeed. It's also the reason we had the dot-com boom in the first place: there were a million ideas that might work, and somebody tried every one of them. Of course, there were also a million ideas that were idiotic and DOA, and somebody tried those too...
By contrast, the use cases for LLMs so far seem to be
- badly-written low-value code
- chatbots that prevent you reaching a human helper
- six-fingered revenge porn
Some thoughts on the dot.com bubble comparisons
First mover advantage, Nvidia/OpenAI remind me of Netscape, Yahoo, Cisco and Nortel, only one of those 4 is really left standing
Money, dot.com was at a time of booming economy, lots of easy money at low rates, AI is at a time of less easy money (still relatively easy) but higher interest rates. Also dot.com companies had very little revenue, Nvidia posted 26Bn in Q2
Beneficiaries, dot.com was thousands of startups splitting the money, AI is maybe 10 and they hold the majority of the patents too.
Infrastructure, in dot.com bandwidth and servers were toy like in comparison to today, AI is more about power issues.
Assuming that AI pops it's bubble like dot.com we still got lots of fiber and datacenters that went on to service the post dot.com internet. If AI goes the same way there is going to be a lot of compute available at relatively cheap cost.
And post dot.com left companies like Amazon and Google standing tall, they at the time weren't exciting unlike pets.com and Webvan. Remember those? Post AI, identifying the likely winners today, might need a crystal ball, could be an interesting exercise. I'd be looking at utility companies, Synopsys and Cadence FWIW
It is a much bigger niche than NFT, Metaverse and VR, but yes a niche and a massively overvalued / overfunded one at this time compared to what it is able to do, based no doubt on the hope or FOMO from promises/claims about "AGI". None of this can possibly ever lead to AGI. If we ever get there, it will be by something as totally different from LLMs as LLMs are from their "machine learning" predecessors 25 years ago that struggled with speech recognition and promised language translation of halfway passable quality in a decade.
I think you are right, AGI will be some leap in thought or approach, not iterative improvements of LLMs. And for all that the AI hype sucks, there are some use cases where advanced pattern matching is producing useful results. Not necessarily innovative but in materials science, pharma and drugs it is capable of scouring far more information and permutations than a team of humans, highlighting the more likely success paths and giving research teams leads to new and exciting discoveries.
That type of scenario is, IMHO, ideal usage of LLMs. Curated data fed to a pattern matcher, flags interesting patterns or areas worth exploring, human subject matter experts review the findings, whittle them down to the most likely results and then test the results. It works as a more general expert system in that scenario, and the lack of polluted data sources keeps halucination to a minimum and the results are verified by humans for more exploration.
I think of this as like the autopilot in a plane, sure it could be left to fly the plane on its own but you don't. The pilots use the autopilot to reduce the drudgery and monotonous operations and are the ultimate decision maker if things get out of control. Well, unless you are Boeing and override the pilot inputs because your algorithm knows best and then don't tell the pilots this is a possibility, see MCAS :(
Nvidia just wants to sell GPUs to whomever. At the moment OpenAI has some momentum but big challenges with $$ of course, no reason for Nvidia to commit anything to OpenAI in fact it seems they have a massive OUT of whatever agreement when OpenAI fails to find the $$ for data center space/power/cooling/etc to run this stuff. Then Nvidia will sell to others. No reason to think OpenNvidia will dominate anything in 2040s or any time, there's too much competition out there with much deeper pockets than OpenAI both on the software and on the hardware front that are panicking over FOMO.
"where exactly will all the other AI software companies get the Nvidia GPUs
No need for Vera Rubin processors - I am sure the UK Government will rise to the occasion and lob about £10,000 to the University of Manchester to develop a more powerful GPU - The Vera Duckworth
