Failed to open management RPC connection

[osonder]

Support case id: 00189706

Have a Veeam server running 6.5.0.128 All backup is running to a Backup Repositories on an other server (named Mngt01) with IP 192.168.133.162

On this server I removed and old installation of vSphere Client and Veeam. After this, backup is no longer succelful. All jobs is failing with this error:

15.03.2013 18:24:16 :: [192.168.133.162] Failed to open management RPC connection to proxy service. Port: '6162'. The RPC server is unavailable RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [192.168.133.162].

I have done following to try to correct the error:
- Rebooted the Veeam server and the Mngt01 server
- Deleted and recreated the Repositories.
- Rescan Repositories is completing without errors.
- Deleted backup job and created new
- Backup to local Repositories is OK

[jeffmetcalf]

I have started getting this exact same errors on all of my backups. Situation on mine is that I replaced a repository.

[osonder]

I was able to solve this error. The repository server was missing the Veeam Backup Transport software. This piece of software is beeing installed when you set up a totally new repository. Since I had this server already added as an IP adress, I created a new repositorie with the Name of the server instead. Then the Backup Transport software was re-installed.

[Bolo]

Hi,
I am having same issue with one of clients environment. I did all the actions listed by osonder, however it did not help. I have uninstalled windows updates corresponding timely to when backup started failing, it seems to have fixed the issue(its still ongoing but at least shows progress whilst before it failed before showing progress)
The updates were on 15/07, does that ring any bells to anyone?

[Bolo]

Hi,
I am fairly new to veeam but already have stumbled upon quite problematic issue.
For few of our clients backup job started to fail from time to time with this error:

Code: Select all

Error: The remote procedure call failed and did not execute
RPC function call failed. Function name: [InvokerTestConnection]. Target machine

Backup server is windows 2012, virtualized through vmware
After a while of checking what could be the source it appeared to be particular windows update:https://support.microsoft.com/pl-pl/kb/3067505 as this one is designed for security purposes of using RPC for gaining access to machine
So after uninstalling this particular update from Backup server (all updates can be installed up to date on backed up machines with no problem) backup started working correctly.

Now, I hope this might come in handy to some people, however I would really like to have this loophole somehow covered, so that an outside attacker will not get through by using this particular missing update.
Any thoughts?

[Shestakov]

Hi Boleslaw,
RPC function call failed is a pretty generic error. Maybe the update closed some ports used by Veeam Backup & Replication components.

Could you please open a case with Veeam technical support team and post its number here for us?
Also, do you use default Windows firewall settings?
Thanks!

[Bolo]

We tried with firewall off to see and it did not help, only after uninstalling this update backup started working. My colleague opened the case when I will know the number I will post it here

[Shestakov]

Thanks for the reply,
Another possible root is VeeamVSSSupport service in the Guest OS. Troubleshooting is stated in the related KB:
When no job is processing the VM connect to it and check the following things:
1. Make sure the VeeamVSSSupport service is no longer present.
•If it is use the command ‘sc delete VeeamVSSSupport’ to remove the service.
2. Make sure the c:\Windows\VeeamVSSSupport folder is not present.
•If it is delete the folder. If it fails to delete reboot or resolve the file lock.

If you don`t have the update installed, you can`t check if it was the reason though. The best way is to check logs with the support.

P.S. There is also a Microsoft KB article with additional troubleshooting.

[Bolo]

Hi,
I dont think veeamvssuport is the issue. I will go through microsoft kb article and additionaly will go through ports, what I think happens is one of port in dynamic range gets blocked and that would actually explain why sometimes it worked and sometimes it didnt after installing the update

[Shestakov]

Let me know if you have any success troubleshooting the issue by yourself. Also try to check antivirus exclusions.
And again, once you have a support case number, please post it here.
Thanks!

[profbma]

Error: The RPC server is unavailable RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [x.x.x.x].

It's definitely the absence or wrong version of one of the three components installed to a windows server that is a backup target.
1) Veeam Backup Transport
2) Veeam Installer Service
3) Veeam Backup vPower vNFS (only gets created when creating repository with the check box next to "Enable vPower vNFS")

I encountered the error after I upgraded to v8 from v6.5. I eventually had to manually uninstall the components on the repository windows machine, they wouldn't upgrade during the upgrade install of v8.

I continued getting this message until I removed all backups, repositories and windows servers and started over by creating a windows server as backup target, creating repository (on that windows server) and then recreating the backup job using that repository. A bit extreme if you have a lot of backup jobs to that backup target. The installing as the name instead of IP address trick (mentioned by osonder) works great except I used the trick before I manually removed the v6.5 components from the target windows server so I was still hosed up until I removed everything and started over.

The other RPC error people are referring to, error 1722, has to do with dynamic port allocation or firewalls blocking application access or virus scan blocking application access. That's not what the original poster was dealing with however.

[AWC]

Support ID 01044814 - Backups Failing Frequently

Error Msg : 17/09/2015 20:55:23 :: Error: The remote procedure call failed and did not execute RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [192.168.**.**].

Thank You for this information for us it was the Windows Update KB3067505 https://support.microsoft.com/en-us/kb/3067505 I uninstalled this update today and for the first time since it was installed (21/7/15) all of our backups completed first time with no errors. But I need to reinstall the windows update as it is an important security update so has Veeam got a patch/fix for this yet ? or has anyone found a workaround other than unistalling the windows update ?

I have a support case with Veeam and have tried many things they have suggested but none have worked.

[VladV]

Support ID 01044814 - Backups Failing Frequently

Error Msg : 17/09/2015 20:55:23 :: Error: The remote procedure call failed and did not execute RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [192.168.**.**].

Thank You for this information for us it was the Windows Update KB3067505 https://support.microsoft.com/en-us/kb/3067505

Thanks for this tip. We were seeing the following error Error: The remote procedure call failed and did not execute RPC function call failed. Function name: [GetSvcVersion]. Target machine: [10.x.x.x]. after we installed the update you mentioned. So I have the same question, is there a fix for this?

[foggy]

I believe there is some update available through support that addresses several RPC issues, worth a try to ask for it your engineer.

[iwik]

Ah, it seems I'm not the only one getting this error

Code: Select all

Processing XXX Error: The remote procedure call failed RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [10.X.X.X].

Our veeam server is connecting to veeam proxy servers in remote locations using vpn (ipsec). In backup jobs for remote locations backup of some vms fails with this error. After failure automatic retry is successful, so backup is working (but with warning).

In veeam logs I have found it just can not connect. I don't know true reason why and it is difficult to troubleshoot this error. (OS or network problem?)

[foggy]

Have you provided your logs to support for review?

[slos]

Case 01126876; logs submitted.

Have a nearly identical issue that is also resolved by removing KB3067505. This is not a backup issue in my instance but a restore annoyance as the KB needs to be uninstalled and the Veeam Server rebooted before FLR can occur.

My rendition of this event has occurred with more than one site with totally separate infrastructures.

What is specific to my instance is that is issue is only experienced with WinServer2012R2; similar infrastructure stacks are constructed with Veeam installed to WinServer2008R2 server and the issue is not reported at this time. The 2012R2 setups use different proxy types [Network & HotAdd] but both use a CIFS/SMB share as a repository . Veeam8 is SP3 at all locations.

Further Veeam KB searching took me to: http://www.veeam.com/kb2065 This page notes:

Cause
The issue is related to an RPC change introduced with a Microsoft security update KB 3067505 https://technet.microsoft.com/en-us/lib ... 5-076.aspx.

Solution
You can obtain a hotfix for Veeam Backup & Replication version 8.0 by contacting Veeam Support. Please refer to issue 54622.
For version 7.0 the only workaround is to uninstall update KB3067505 from Veeam B&R server.