There is always a possibility of something like this happening when you leave remote control server directly accessible from the internet. If you need 24/7 remote control access you should run OpenVPN or similar VPN server and allow outside connections only through the VPN tunnel.
Ironically, the very technique (hole punching to circumvent routers) that makes Teamviewer faster to set up the normal way makes it more involving to set it up a safer way that allows access only through VPN.