Host-based backup of VMware vSphere VMs.
Post Reply
unsichtbarre
Service Provider
Posts: 226
Liked: 39 times
Joined: Mar 08, 2010 4:05 pm
Full Name: John Borhek
Contact:

Veem B&R 8 seems to take much more time processing replicas

Post by unsichtbarre »

We recently upgraded to Veeam B&R v8 and have noticed that it is slowing down our replication jobs significantly. We have gone from surpassing a 30min RPO to requiring greater than 2 hours/job

The slowdown happens during the step(s) "Using target proxy Blah Blah Blah for disk Hard disk X [nbd]" and holds the job for 4-12 minutes per VM. We have parallel processing enabled and enough proxies/CPUs to run multiple disks at a time - and even when only one VM is processing, there is a tremendous delay we did not see before.

If you add 4-12 minutes per VM on a job with 10 VMs/replicas, we are no longer to meet a RPO less than 2 hours, whereas in Veeam 7, we were easily able to meet sub-30 min. RPO

After the data starts to replicate, replication it seems to move as fast as it ever did.

Any ideas?

This case has been sent to Veeam Support and log files have been uploaded.
Case #00733966

Thanks,
John Borhek, Solutions Architect
https://vmsources.com
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by Vitaliy S. »

Have you tried using hotadd backup mode at the target location to see if it makes any difference or not?
unsichtbarre
Service Provider
Posts: 226
Liked: 39 times
Joined: Mar 08, 2010 4:05 pm
Full Name: John Borhek
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by unsichtbarre »

We have been burned by hotadd in the past. If possible, we would like to avoid it.

Also, v7 was not configured for hotadd, prior to the upgrade.

THX for the response.
John Borhek, Solutions Architect
https://vmsources.com
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by Vitaliy S. »

Ok, let's see what our support team figures out after reviewing the debug logs. BTW, does this happen on all target proxy servers?
unsichtbarre
Service Provider
Posts: 226
Liked: 39 times
Joined: Mar 08, 2010 4:05 pm
Full Name: John Borhek
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by unsichtbarre »

At this point, there are two proxies at source side w/4 vCPU each and only the Veeam server at target side w/8vCPU. I can add an additional proxy at the target and test, although the problem occurrs even when only one VM with one disk is running.

The support team has responded and said that "it looks like encryption in enabled." We have double-checked and encryption is not enabled between host/proxy. Since it is a replication, we dont know of any other encryption feature.
John Borhek, Solutions Architect
https://vmsources.com
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by veremin »

Most likely, they're talking about traffic encryption that can be found in the global throttling settings. Thanks.
unsichtbarre
Service Provider
Posts: 226
Liked: 39 times
Joined: Mar 08, 2010 4:05 pm
Full Name: John Borhek
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by unsichtbarre »

We are still struggling with a halt of about 6 minutes per VM, per job.

Encryption was never enabled anywhere in the GUI, but on the advice of Veeam support we have applied the following Registry key:

DisablePublicIPTrafficEncryption

Type: REG_DWORD
Default value: 0
Description: By default, traffic between different subnets and on public addresses is encrypted. To globally disable network encryption between agents, set to 1.

In the following image, you can clearly see where Veeam just stops for over 6 minutes. In Veeam 7, this job was finalizing both VMs in 3-5 minutes. In our other job with 10 servers, we are adding 60 or more minutes, just waiting around

New logs have been gathered and uploaded to the same case referenced before.

Thanks,

Image
John Borhek, Solutions Architect
https://vmsources.com
unsichtbarre
Service Provider
Posts: 226
Liked: 39 times
Joined: Mar 08, 2010 4:05 pm
Full Name: John Borhek
Contact:

Re: Veem B&R 8 seems to take much more time processing repli

Post by unsichtbarre » 3 people like this post

Thanks to Ken Sauer at Veeam Support, this is now resolved.

Network Connections > [ALT] > Advanced > Advanced Settings > Adapters and Bindings

Place the Replication network first in the list and the local vSphere Management Network second

Thanks Ken!
John Borhek, Solutions Architect
https://vmsources.com
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 98 guests