"So "simple" (not!) smaller features and larger device-counts seems to beat "smarts" all hollow."
You don't appear to have heard of "Big O " notation. :-(
If the size of the problem is big enough it will still bring the SoA hardware to its knees. That's where this comes in. The problem size you can tackle with better algorithm goes up regardless of how fast the cores (or how many of them) there are to begin with. It's like the Fast Fourier Transform's effect on signal processing software. Turning overnight runs into minutes of computer time.
CFD problems (for example) use lots of matrix math and continue to bring SoA HPC arrays to their knees.
Biting the hand that feeds IT
