Comprehensive data protection for all workloads
Post Reply
Wtrippler
Lurker
Posts: 2
Liked: never
Joined: Jan 12, 2018 1:58 pm
Full Name: Wendy Trippler
Contact:

Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by Wtrippler »

Since upgrading to 9.5 update 3, we are getting this error when trying to access our data domain devices:
Failed to call RPC function DDBoostSetCreds

Support fixed this once by restarting Veeam services and running the VeeamPluginsHostX64 application from C:\Program Files (x86)\Veeam\Backup Transport\x64

It was fine for 2 days then all of our jobs failed last night with the same error. Any ideas?

Thanks!
Wendy
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by veremin »

Kindly, re-open ticket with our support team and let them investigate it further. Thanks.
javis
Lurker
Posts: 2
Liked: never
Joined: Jan 19, 2018 3:43 pm
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by javis »

We also are experiencing this exact same error with out data domain after installing 9.5 update 3. We have a ticket opened with Veeam support.
rntguy
Enthusiast
Posts: 82
Liked: 1 time
Joined: Jan 29, 2016 8:31 pm
Full Name: Rnt Guy
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by rntguy »

We have the exact same error. Any luck with this?
rntguy
Enthusiast
Posts: 82
Liked: 1 time
Joined: Jan 29, 2016 8:31 pm
Full Name: Rnt Guy
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by rntguy »

Wtrippler wrote:Since upgrading to 9.5 update 3, we are getting this error when trying to access our data domain devices:
Failed to call RPC function DDBoostSetCreds

Support fixed this once by restarting Veeam services and running the VeeamPluginsHostX64 application from C:\Program Files (x86)\Veeam\Backup Transport\x64
How did they 'run' that? I just double clicked it but not sure what to look for.
javis
Lurker
Posts: 2
Liked: never
Joined: Jan 19, 2018 3:43 pm
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by javis »

This issue had to do with our remote proxy server which is on a different subnet. We've rebooted our Data Domain, proxy server, Veeam server and still didn't solve the issue. I noticed our Veeam server could not establish a connection to our remote proxy on port 6160. I restarted the Veeam Installer Server (our version is 9.0.0.1715) on our remote proxy server and connection was established on port 6160 and this has been working great ever since.
tkeith
Enthusiast
Posts: 32
Liked: 17 times
Joined: Jan 09, 2015 4:49 pm
Full Name: Keith Thiessen
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by tkeith »

We are having similar issues and have missed backups the last two nights!
All our issues started after we updated to Update 3 last tuesday...

Getting different errors:
Error: Failed to call RPC function 'DDBoostFcIsExist': The remote procedure call was cancelled. RPC function call failed. Function name: [DoRpc].
Error: Failed to call RPC function 'DDBoostSetCreds': The remote procedure call was cancelled. RPC function call failed. Function name: [DoRpc].
Error: The RPC server is unavailable. function call failed. Function name: [GetSvcVersion].

These error happens against the data domain, mount servers, and now the backup server itself. We seem to have the installer service crash

We back up thousands of VMs and this can't be happening!

Currently under Veeam support case: 02464083
omegagx
Enthusiast
Posts: 68
Liked: 4 times
Joined: May 09, 2017 6:33 pm
Full Name: Mike G
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by omegagx »

Did you ever get this resolved?
UFE
Lurker
Posts: 1
Liked: 1 time
Joined: Mar 05, 2018 6:52 am
Full Name: IT team
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by UFE » 1 person likes this post

Hi there,
we had a nearly same problem.
Error: Failed to call RPC function 'TestCompatible': Error code: 0x80070583. Cannot initialize COM runtime.
here is the solution from Veeam support

"It is much pretty common with Update 3 when 3rd party monitoring tools proadcastrsVeeam with PowerShell queries and it affects our backups.
I would recommend the following steps:

1) Disable zabbix service
2) Restart Veeam server
3) Make sure that service is disabled
4) Open Veeam and re-try the job

You can use SNMP instead of Powershell as a workaround or refer to 3rd party software in order to have latest updates

Please keep me updated. I will be happy to help or answer your questions. Have a great day!
Best regards,
Mikhail Zolotov
Veeam Support"

After disabling Zabbix agent, backups works normally. Do you have some monitoring tools there?
Sindhuja
Influencer
Posts: 21
Liked: never
Joined: Jan 22, 2018 11:01 pm
Full Name: Sindhuja Balasubramanian
Contact:

[MERGED] Failed Backups due to execution of Powershell queri

Post by Sindhuja »

Hello All,

We observed a weird issue at one of our customers, possibly related to a combination of Veeam backup & replication 9.5 Update 3, Data domain and Powershell scripts.
All backups (which were successful so far) failed after the execution of our utility that queries Veeam Server using Powershell Cmdlets (as exposed by VeeamPSSnapIn).

Code: Select all

"Error: Failed to call RPC function 'DDBoostSetCreds': The remote procedure call was cancelled. RPC function call failed. Function name: [DoRpc]."
After some lookout, we came across the below thread,
https://forums.veeam.com/veeam-backup-r ... ml#p273664

This thread mentions, "With Veeam Backup & Replication 9.5 Update 3, the use of Powershell Cmdlets queries may result in a backup failure".
This is very unfortunate!

We would like to know the following:
  • Is this a known issue and has it been communicated to the customers?
  • Is there a patch available to fix this issue? If not available, any ETA on the patch?
  • Is this related to any specfic Powershell Cmdlet which has an alternative approach and can be avoided?
Regards,
Sindhuja
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by veremin »

Few questions to start with:

* Have you already reached our support team?
* Have you tried the workaround mentioned above?

Thanks.
omegagx
Enthusiast
Posts: 68
Liked: 4 times
Joined: May 09, 2017 6:33 pm
Full Name: Mike G
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by omegagx »

This is exactly the type of issue that should have a knowledge base article written up about. It would allow the customer to find a solution (disable monitoring such as zabbix) on the Veeam website instead of wasting time calling support. If this is a known issue that was already experienced by other customers, why not document it instead of forcing everyone to call in?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by foggy »

Support has a hotfix for this issue, so please contact them to get it.
Sindhuja
Influencer
Posts: 21
Liked: never
Joined: Jan 22, 2018 11:01 pm
Full Name: Sindhuja Balasubramanian
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by Sindhuja »

Thank you for the response!
We will reach out to Veeam support to help us with the hot fix.
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by Gostev »

omegagx wrote:This is exactly the type of issue that should have a knowledge base article written up about. It would allow the customer to find a solution (disable monitoring such as zabbix) on the Veeam website instead of wasting time calling support. If this is a known issue that was already experienced by other customers, why not document it instead of forcing everyone to call in?
Simply because it did not pass the threshold on the number of support cases opened. With up to 1000 support cases open daily, obviously they have to approach KB article creation selectively. Thanks!
oncominaspa
Lurker
Posts: 2
Liked: never
Joined: Mar 16, 2018 12:41 pm
Full Name: TI Pouso Alegre
Contact:

Re: Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by oncominaspa »

We have the same problem here.

After upgrade, only two jobs executed well, and then: Error: Failed to call RPC function 'TestCompatible': Error code: 0x80070583. Cannot initialize COM runtime.
If we restart the Veeam Backup Service, more two jobs execute well, and the third fail again.

I will try to disable the zabbix agent.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by foggy »

I'd contact support to verify the issue and get the hotfix.
omegagx
Enthusiast
Posts: 68
Liked: 4 times
Joined: May 09, 2017 6:33 pm
Full Name: Mike G
Contact:

Re: Veeam 9.5 Update 3 RPC Error

Post by omegagx »

What is that threshold? I think that *ANY* issue that requires an actual hotfix because Update 3 broke something in terms of functionality ABSOLUTELY requires a corresponding KB article. Unless you already released over a 1000 hotfixes for 9.5 Update 3 but I sincerely hope that's not the case.
Gostev wrote:Simply because it did not pass the threshold on the number of support cases opened. With up to 1000 support cases open daily, obviously they have to approach KB article creation selectively. Thanks!
oncominaspa
Lurker
Posts: 2
Liked: never
Joined: Mar 16, 2018 12:41 pm
Full Name: TI Pouso Alegre
Contact:

Re: Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by oncominaspa »

oncominaspa wrote:We have the same problem here.

After upgrade, only two jobs executed well, and then: Error: Failed to call RPC function 'TestCompatible': Error code: 0x80070583. Cannot initialize COM runtime.
If we restart the Veeam Backup Service, more two jobs execute well, and the third fail again.

I will try to disable the zabbix agent.
After disable the Zabbix agent service, the VBR works ok!

PS: I think that is a issue with the update 3 and Zabbix agent. I hope Veeam team can resolve soon this issue! Until that, our VBR server monitoring with zabbix is disabled.
jah2323
Novice
Posts: 6
Liked: never
Joined: Sep 27, 2012 7:22 pm
Full Name: Jason A Hassett
Contact:

Re: Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by jah2323 »

awesome! as i receive the error at random times and on differing veeam servers. it is usually related to windows updates, so i thought, and uninstalling the last one would usually fix it. weird that this time, the server waited weeks to have the problem, and it only showed up as windows 2016 has been telling me it has updates to install. I will get with support here shortly.
zag
Novice
Posts: 3
Liked: 2 times
Joined: Mar 04, 2016 4:50 pm
Full Name: Zag
Contact:

Re: Veeam 9.5 Update 3 RPC Error with Data Domain Boost

Post by zag » 1 person likes this post

I just got this error as well, after a good call with support we found the DLL in C:\Windows\Veeam\Backup and Replication on the host was corrupted or had reverted to an older version.

Re-installing the agent on the host fixed this issue.
Post Reply

Who is online

Users browsing this forum: Google [Bot], marcin.dudziak, Markush-VE, mbjr, xzvf and 158 guests