Re: "I don't always need to look up the address of a bit of kit I need to contact"
It's not really niche when it has almost two billion users. v6 is just hard to scan because it's so huge. Exhaustively scanning one /64 requires something like a million terabytes of traffic. If you're on a regular, slow 10 Gbit/s link that'll max it out for thousands of years.
(Obviously there are methods of finding hosts that aren't exhaustive scans... and people do use them on v6. You just won't see a constant stream of opportunistic scan attempts on every single address for every minute of every day.)
IPv4 addresses aren't exhausted, just their allocation is. a good 90% of them just aren't seen on the internet.
When we say exhausted, we mean that people have to take measures to conserve addresses due to not being able to get enough of them. The beginning stages of exhaustion happen when people start needing to do that. We're deep, deep into it for v4.
Also, due to the way hierarchical aggregation and routing works, most individual IP addresses don't end up assigned to actual end devices, so "I haven't seen any internet traffic from 90% of addresses" doesn't mean that v4 is only 10% used.
I'm also going to point out that there are more individual devices than there are v4 addresses, so there's not going to be enough no matter how you slice them.
TBF if the internet core was run on v6 with BGP and everyone used NAT and RFC1918 addresses internally, there is no need for the added complexity of v6 inside an organisation
You mean the reduced complexity... and you would in fact need it, since v4 doesn't handle more than 32 address bits.
the NHS internally uses the RFC 1918 class A space and it hasn't exhausted it yet.
But the fact that they're using it at all is an indication that v4 is insufficient. Why go to all the added complication otherwise?
IMHO when they designed v6 they went overboard on the address space
Absolutely! Overboard is good. We want to have more address space than we can use, because the alternative is that we'll end up using it and will need to deploy another extension. As you might have noticed, that's a very difficult, expensive, long and drawn-out process that we really only want to do once.