Re: yeah but
"If you didn't build it yourself how well do you know the person who built it for you?"
At some point, you have to trust someone - be that the team behind Arrow (my current 10-based ROM), though Cyanogenmod / Lineage in the early days, all the way to Google for the AOSP source in the first place... none of which I know personally, but trust to create something suitable for my needs. (I wasn't suggesting installing any random junk, by the way)
"It's not a simple task to build Android to run on something, nor is it simple to figure out how to flash it unless the bootloader is very accommodating."
Agreed on the build process...
...the flashing process, however, is not as arcane as once it was given that there are PC-based tools that will (bootloader permitting) automate the tasks required, reducing it to the few button clicks needed to download a suitable file, reboot the handset into the proper state, and then flash the new/updated ROM ... from memory this is exactly what the Kies Manager did for Samsung handsets back in the day (but it's been a while since I last owned one) - or iTunes for iOS devices.
The Project Treble initiative should make the process simpler still (or that's the impression I get from https://developer.android.com/topic/generic-system-image/ ) given a suitable (modern) handset. Whether it'll exist in the long term is one for the bespoke/generic debate I guess.