d-Matrix bets on in-memory compute to undercut Nvidia in smaller AI models

Generative AI infrastructure builds have given chip startups a hardware niche yet to be targeted by larger players, and in-memory biz d-Matrix has just scored $110 million in Series-B funding to take its shot. d-Matrix's cash came from investors including Microsoft and Singapore's sovereign wealth fund Temasek. Founded in …

[that one in the corner]

Fairly short window of opportunity

LLMs are very much flavour of the day, in much the same way as everyone talks about the latest fantastic bridge project (the press around the building of the Clifton Suspension Bridge was vast and colourful[1]). Meanwhile, there are many more people are being regularly served by all the hundreds (thousands?) of smaller footbridges and unglamourous overpasses put up in the same timescale. Similarly, for each honking great LLM there are numerically many more tasks that can be tackled with smaller ML models.[2]

If d-Matrix can avoid getting sucked totally into the vision of selling increasingly large systems to make a small number of big sales, before being overrun by the likes of nVidia, there should be a healthy long-term market for decent model-running chips that can be used in smaller products, that sell in larger numbers. Some alternatives/competition down there would be a Good Thing, especially if it can bring down prices and become well-known outside of specialist circles. Okay, they are still up against nVidia (Jetson) but that is put at the pricey end, leaving much room below where there is, what, the K210, anything else?[3]

[1] actually, mostly b&w 'cos the newspapers back then - look, figure of speech, ok!

[2] not really an amazing observation: so long as it is possible to build a smaller working X you will see more mini-X than maxi-X in use: consider JCBs and the cute little mini-diggers.

[3] genuine question, do you know of other devices in that range or lower that help run your smaller ML models? Actually available as hardware, not just drop-in IP for your next fab run?

[Korev]

Re: Fairly short window of opportunity

Quite, I'm involved in a project building chemical models, we only normally need to train on a single GPU...

[Bitsminer]

Re: Fairly short window of opportunity

Some other devices (not a complete list, of course):

groq makes a PCI card and a server groq.com (note that is a Q not a G!)

mythic has some cards that are a bit old now mythic.ai

anari does an FPGA-based solution on Azure

And without looking hard I suspect there is IP for FPGAs that you could apply. But the integration effort to connect your software to an FPGA is huge.

I suspect integration for any kind of hardware is going to be difficult and time consuming.