VPN is indeed the way to go. What really would be a nice feature is to define an alternate IP address in the backup job configuration. When you use local ip address the job uses the internal LAN address. When using a VPN solution (E.g. PPTP VPN) you use the assigned IP address in the backup job configuration for the VPN tunnel.
Managed to test this with incoming connections enabled on a Windows 2012R2 server