Doesn't affect IAX2
SIP n RTP are only one way to "do" VoIP. There are other protocols available.
Also, if you consider VoIP traffic as having a potentially serious value - eg if someone breaks in and gets your PBX to dial their premium rate number - then you treat it as such. VPNs, encryption and firewall allow rules from/to suppliers/partners etc should be the norm.