What an amazingly bad idea by OpenAI
As a person who regularly investigates accelerators for AI, Intel is the most opaque company in the AI industry.
So here's how it works.
1) Student at the university is studying AI
2) Student is given a project and investigates the tools available to them to implement it.
3) They settle on PyTorch or another toolkit for their project
4) They start coding on their laptop or gaming PC
5) They request resources from the university HPC department
6) They complete their project
7) They enter the industry
8) They prefer to use the tools they studied and develop systems professionally as they did in the university
This has been the story of all Intel failures over the past several years.
Intel released piles of new technologies. Some of them far better than their competitors and able to accelerate some tasks by 1000x or more over using general purpose computing. These technologies were absolutely amazing. And they ALL FAILED.
The reason is simple.
University students, hobbiests, researchers, etc... don't have access to Intel's hardware and technologies.
But wait! Intel lets you download the software! That's good right?
Well, the software is useless without the hardware.
Here's where AMD killed Intel.
Whether you're coding for a laptop or a super computer, coding for AMD CPUs is the same. Of course, it's not nearly as amazing as using specialized CPU cores or fancy encryption engines and whatnot. But, the students have access to the technology they need wherever they are. In fact, coding for an AMD super computer can be easily achieved from a Windows WSL2, Mac or Linux session on an old laptop running an Intel chip. Coding against an Intel super computer will not be nearly as optimal because Intel forsaked high performance HPC cores for specialized cores that no one codes for.
Here's where NVidia absolutely destroyed Intel.
Every kid these days has access to a CUDA capable GPU either in their laptop or gaming rig. Coding for an HPC loaded with NVidia H200's (I'm using one with 300 of them right now), is precisely the same as coding for my laptop which has an old RTX4060 in it. The laptop also has Intel Iris XE graphics. But, if I want to code for Gaudi, I can't use the Iris XE for that.
So, that brings us to developer support.
Intel has absolutely no interest in working with the little guy. In a perfect world, what intel would do is sell a student kit containing a ARC B580 12GB for a deep discount and ensure that all Gaudi development tools worked with the card as a first class citizen.
Let's say this, I'm about to spend about $200-$300 on a used NVidia card. On that card, I'll focus my AI development using CUDA and NVidia development tools.
If I could buy a brand new ARC B580 with similar performance to the used NVidia card and I could run the Gaudi development tools on it, I would consider that instead.
Another option, I just looked at Intel's developer support for accessing Gaudi in the cloud.
1) It's offered by IBM/Redhat. There is no way in hell I'm going to build on that platform. I'm not a bank. Only banks and major financial institutions buy IBM stuff. That's not for the small-fry.
2) I couldn't afford it anyway, I followed the link and apparently, it's like $43000 a month. That's not a made up number, it's what the website for Gaudi on the IBM cloud says.
In fact, here's a secret, when I have access to NVidia pretty much everywhere and I know that NVidia works and is proven and I can get support from just about everywhere, I'll drop money on NVidia. But Intel is unproven. At the HPC conferences when you ask Intel about Gaudi and are even willing to accept less performance to see if building systems on it could have potential, the sales people either ignore you or treat you like you're pathetic if you want to even look at their pathetic solution.
See some of us only have $10 million or so to spend on AI accelerators per year. With that budget, you have to wait 24 months for delivery from NVidia today.
But if Intel has a stockpile and capacity to build more, giving them $10 million today can get you cards tomorrow. They apparently are only as fast as the card NVidia can deliver today as well. They don't match the cards you order today and receive in two years. But in two years, I'll have another $10 million to spend and would prefer to not wait two years for whatever the new stuff from NVidia is them.
Then there's the other HUGE, MASSIVE, IRREDEAMABLE failure from Intel.
If you ask for information on Gaudi, rather than asking "What's your address? We'll send you an presales team!!!", they tell you "Talk to our partner Arrow".
Seriously!!!!! WTF?!?!?!?!
It took NVidia 3 days to send a team to me in Norway to have a discussion. A sales person and two engineers where there in an instant.
The funniest thing was, I almost begged Intel to talk with me. They insisted I had to talk to a company I have no relationship to. Not only that, I can't order from Arrow. We have framework agreements which last years and Arrow isn't one of them. I would be asking Arrow to invest time and money into making a sale for Intel even if they had no possible opportunity to make money from it. Do you think Arrow would want to talk with me based on those conditions? Not only that, but Arrow certainly doesn't keep a team of skilled Gaudi engineers in Scandinavia.
So, when Intel says that they are having problems selling Gaudi, let's make it clear.
Their problem isn't finding customers. Their customers are happy to try and find them. Their problem is telling the customers to piss off.
So, Katti might be one of the worst procurements in the history of humanity. He has absolutely no idea about technology. He has no clue how technology works. He basically took one of Intel's most promising projects in decades and destroyed it.
Intel, if you're reading this (I'm sure at least one person there is), get your shit together. Start seeding the developers. Get the students up and running. Get the tech in the hands of the people who are interested in trying it. I have no idea what it costs you to actually make a Gaudi 3 card, but make a pile of 5000 of them and send them out as Christmas gifts to absolutely anyone you think should try them.
BTW, I have a Intel Xeon Phi sitting on display on a shelf in my office. This was one of those gifts from Intel... they showed up at my door and said "Here, try it" and my colleagues and I did. We then build a super computer on them. Then just as all the students who used that HPC and the Xeon Phi graduated, you pulled the damn plug and they had to learn NVidia instead.