Is CUDA still the real Nvidia moat?
I agree that CUDA was the moat back in the days we called these things GPUs. In those days we felt processing graphics and videos and smaller scale forms of AI was a lot of data and processing. Now with LLMs, the scale of processing is a new ball game. Nvidia's new moat has everything to do with scale that often saturates Ethernet and melts components. New highspeed memory chips and joining multiple processors on the same board or even the same silicon. Liquid cooling, DPUs, infiniban and who knows how many trade secrets, patents, strategic partnerships are needed to get the job done right. Imagine the patents owned by the creators of the robots and automated precision manufacturing required to assemble these things and sourcing reliable suppliers. Don't forget the engineers who actually have the skills and experience at building something that resembles a super computer and other aspects of HPC. As an example, many of us are programmers here but few of us are kernel programmers. I think CUDA is still part of the moat but keep in mind they are building a high performance fighter plane and not just an engine.
Biting the hand that feeds IT
