Re: Just check out the paper
Software cached just sounds like what IBM did with the Cell processor, where the data and code in the SPE's limited RAM was not controlled by a hardware cache mechanism second guessing what the software would ask for next, but by the orchestrating software putting data and code there for itself.
From the point of view of developers like me, that was fantastic. However the problem was that it was so far off the beaten track for a lot of developers it bombed as a concept and IBM dropped it, which was a real pity. Neat as this latest idea is, if it involves custom hardware on IBM's part I'd not be counting on them sustaining it as a dependable product line.
But it's in interesting idea. People have been looking for GPUs that can do more. Trouble is that, taken too far, you end up with something like an AMD APU....