Well...
"On the contrary, it suggests to me that the populace is close to neutral on the issue."
I'm fairly sure that's not what this means. While I'm sure some in either camp are not too fussed either way, feelings are generally pretty strong on either side, even if they average out. Running the risk of being gunned down in the street if you piss of the wrong racist nutter* doesn't add to the appeal.
So if a politician goes strongly one way or another, they can expect to piss off roughly half of their voters. Hence why Cameron and Farage quit, rather than tackle the task, and BoJo was only ever in it for PM, not for any actual work. Farage can't actually build anything, but does a fine line in pissing on things whilst managing a spot of demagoguery.
What makes the whole mix worse is while Remain has a vague consensus on what they'd like, what "leave the EU" means to Leave voters varies a great deal. I strongly suspect when faced with what the actual deal will be, many of the Leave voters will feel betrayed, and rather than blaming the impossible lies they where sold (or where implied and suggested but totally not actually promised), the blame will fall on whoever got the short stick of negotiating.
* I don't think Leave voters are generally more racist than Remain voters, but there are certainly racists who are using it as a smokescreen.
Biting the hand that feeds IT
