Maybe time doesn't progress at a linear rate and time itself progressed faster nearer to the dawn of the universe? We'd see this as things happening "earlier than they should have done".
Perhaps the galaxies took the normal amount of time to form but just look like they formed "too early" due to time at that point running at an accelerated rate? The heartbeat of the universe starting off fast when the universe is more compact, then slowing down as spacetime expands? Maybe the pressure or surface tension of the same amount of planck units existing in a smaller volume of infinite spacetime?
So as spacetime expands and stretches at a faster and faster rate, time is effectively stretched out and "slowing down" now compared to billions of year ago? We wouldn't notice - time would seem to be running the same as it ever did in our present - but it'll be a constant changing throughout the life of the universe?
I expect to be shot down in flames as there's probably lots of evidence to disprove this, but I thought it was a nice idea. :)
TLDR: Maybe what we see as X amount of years years back then is compressed from our perspective.