"it seems to me that the mental retention of literary allusions, cultural references etc is something older people do."
Well that's almost obvious. We've been around for longer. I know *far* more than I did when I was a teenager, about a huge variety of stuff that I didn't even realise (back then) was a thing that you could know about. Given the time-spans involved, it would be a bit embarrassing if I didn't.