For the average AI shop, sparse models and cheap memory will win

As compelling as the leading large-scale language models may be, the fact remains that only the largest companies have the resources to actually deploy and train them at meaningful scale. For enterprises eager to leverage AI to a competitive advantage, a cheaper, pared-down alternative may be a better fit, especially if it can …

[anderlan]

My brain is the average AI shop.

Any Numenta/Nupic boffin will not be the least surprised that sparse representation and cheap memory is the right approach for *most* AI because that's how the individual Natural Intelligence of tens (hundreds?) of billions of mammals does its thing. Only several out of 100 to 1000ish cells fire to represent a thing in cortical memory. And as we age we all know that neurons are crappng out left and right, and still, somehow, we manage to function. These are 2 sides of the same neocortical memory/mammalian intelligence coin.