Reply to post:

I gave up on the NTP pool servers and what not - use the real thing:

NPL time servers

Delay is down to about 18ms and jitter is around 3 ~ 5. Using NTP pool servers, delay could be up to 40ms and jitter all over the place.