Re: Giving it a try
Ccache might help but if just building the kernel takes two hours then you need to invest in new hardware.
I Yocto-build an entire openembedded deployable in about twenty minutes, using locally cached git repos for the various userland porcelain, on a 12-core 24-thread Ryzen, 64GiB RAM & a 2T SSD.
YMMV of course.