Re: Nobody is quite sure what it was originally built for.
Nah, overflow holding for the Dunwich* horrors just up the coast. They calmed down a bit after Sizewell was built near by, thus providing them warm water. Plus HMPs Hollesley Bay & Warren Hill took custody of some of the local ne'er-do-wells.
Which at one point may have included a pre-teen-ish me, who used to spend happy summers kayaking around the Alde. Orford Ness was a kinda odd top secret site given it's proximity to tourists, so also attractive to a young me who wanted to know what the secret stuff was all about. Some chaps in a patrol dinghy politely told me to paddle off, so I did.
You could see the radar installation from the beach, and I may have some pics of it somewhere.. Basically looked like a circle of tall antennas & the kind of thing that could have served as a Mothra pen. Been back a few times as an adult, mainly as there's a couple of landing stations in that neck of the woods & did the tour.. One of the most striking bits was one of the buildings apparently being where the AWE did nuclear bake-offs to see how well they'd last in a fire. Sans physics packages, but sometimes with a fair amount of plain'ol high explosives in place.
Biggest threat nowadays is probably from the locals (ok, rich incomers) who'll complain mightily if you don't park in the right places.
*Dunwich being something of a disappointment when I learned it wasn't the Lovecraft version. Used to have a mighty fine chip shop on the beach, but also an early example of global warming. Dived it a couple of times, but not a lot to see given general water visibility & sediment.