One solution for it is obvious
Just track the eyes, then have 2 screens for your eyes, one that's low res and large for the background, and one higher resolution one which is optically placed at the point where you see the sharpest and at a focal distance that's close to the object you're looking at.
This would dramatically cut back the amount of data you need to generate.