RPC error:The RPC server is unavailable.

[phillbl]

hi,

i have just installed veeam B&R 7.

i have created some test jobs and all seems to be working on most 2003/2008 windows servers. on a few servers i get the following error

29/08/2013 8:15:11 AM :: Error: RPC function call failed. Function name: [IsSnapshotInProgress]. Target machine: [IP address].
RPC error:The RPC server is unavailable.
Code: 1722

I have looked at the Group policy and the firewall rules are present on all servers? anyone know of why it would for some and not other given they have the same firewall rules?

thanks

[veremin]

Hi, Phill. I’m wondering whether you have already tried the solution mentioned in this KB article. Thanks.

[RomanAxBe]

I get on a server with the following message:
"Failed to prepare guest for hot backup. RPC function call failed"



Somehow the backup seems to still be made​​.
Can the backup be used, or which is not good on the backup?

I am really irritated that it will still be made ​​ready.

[foggy]

The first troubleshooting step would be to check whether all the ports inside the VM guest OS required for application-aware image processing are open.

What kind of VM it is?

[RomanAxBe]

It is a Windows Server 2008 R2
It could be because the ports that VM is in a different domain. I check the ports on the firewall

[rurouni]

Hi,
I'm opening this thread, because we have had an incident occuring many times in the past without being able to find clearly the root cause of the problem.

- The incident occurs randomly.
- Backups work fine, one day, the next day lots of job fail, then the day after jobs run fine again.

We opened the case 00598889 for investigation.

We observed that for failing jobs, most vms report this:

Code: Select all

- Unable to truncate transaction logs. Details: The RPC server is unavailable. (Exception from HRESULT: 0x800706BA)
- Failed to prepare guest for hot backup. Details: RPC function call failed. Function name: [IsSnapshotInProgress]. Target machine: [10.29.67.52]. RPC error:The RPC server is unavailable. Code: 1722

Version is: 7.0.0.771

Thank you for your help in the resolution of this case.

Jean-Michel

[veremin]

Can you check whether there are VeeamVSSSupportServices running inside these VMs? Thanks.

[rurouni]

I didn't check them all, but five of the failed vms with the error 0x800706BA have their VeeamVSSSupportService running.

I think that's not it. the problem is more general in my opinion and not related to a specific VM. Yesterday all those vms were backuping fine.

[veremin]

According to the errors provided, most of VMs failed due to RPC unavailability, which in turn might be caused by stalled Veeam VSS service.

However, it's hard to say without deep log analysis. So, let's wait and see what support team says about it.

Thanks.

[DavorS]

Can you check if you can access admin share of that server from your Veeam B&R?
Also, please check in Network and Sharing center -> Advanced sharing settings that you have selected "Turn on network discovery" and "Turn on file and printer sharing" for network profile through which you're accepting a connection on target server.

If all of above works, check that you have the latest Veeam B&R patch (patch 4) applied.

I've seen a few cases where VSS just dumbs out on server 2008R2 because it has too little space on the "system reserved" partition, but when troubleshooting RPC connectivity you should check if everything is ok with time synchronization and DNS entries.

Is it a technical issue that was there from beginning or did it just start one day? If it just started randomly, you can also check what's the group policy password expiration setting? Maybe password for the account you're using to do application-aware processing ran out.

[rurouni]

Thank you four your replies.

DavorS, none of what you suggested were an issue in our case.

I went further and applied Patch 4 before Yesterday evening Backups.(with reboot)
On the other hand I Extended Veeam Backup Server VM from 4 to 6Gb RAM observing that when 12 Jobs are running simultaneously, The 4Gb RAM were completely Used.

I also added 2Gb RAM more on our vcenter which was set to 10Gb for 9.5Gb used.

Today i can say that all the jobs ran normally even if we have a few common errors. 402 of 407vms were backed up successfully. 5 have warnings.

I can't tell the issue is resolved yet, because i had the same behaviour in the past. I'll keep watching it, and update this post if it happens again in a while.

Regards.

[veremin]

Let's see how well it goes for some time. If the issue re-occurs, feel free to re-open your existing ticket and let them investigate it further. Thanks.

[rurouni]

That's it. It occured again.
We ve upgraded to Patch 4. Backups have been running particularily well until yesterday and suddenly some jobs stated warning status with the same RPC error on the backed up VMs.
We updated the case 00598889 with logs.

Thank you.

[veremin]

We've upgraded to Patch 4. Backups have been running particularly well until yesterday and suddenly some jobs stated warning status with the same RPC error on the backed up VMs.

Have there been any changes introduced recently in you environment? What about stalled VeeamVSSservice? Are they still present inside the given VMs?

Anyway, let support team find the root cause, once they analyze the logs provided.

Thanks.

[rurouni]

Support is investigating but my impression is that the only pertinent answer i could get is:

- If the server is not based on Microsoft SQL server, there is no need to set up The Truncate Logs option on VMs which do not require it.

I would agree with that but my opinion is that if you setup a Truncate log behaviour though there is no log to truncate, it should not do any harm.

On the other hand, we observed that When a job fails this way, it usually succeded the previous day, and it also succeeded the next day though we didn't change anything to the job or servers configuration.

That's not a dramatic issue but that makes us uncertain on the reliability of some Veeam Jobs. That's why we would like to find the real root cause to solve it definitely.

Thank you.

[Vitaliy S.]

In order to shed some light on the RPC Server unavailability issue, I would also suggest reviewing Windows Event log (at the time when you saw this failure) with the help of our support team.