Is Veeam Replication direct from physical to physical host?

[Novox]

As part of my normal backup routine, I replicate VM guests from one VMWare physical host to another and vice versa.

• Does the data travel via some sort of VMWare API call or other direct connection between the physical hosts?

OR

• Does the data travel through an intermediary Veeam data mover?

I ask because I wonder if there would be any benefit to Veeam Replication if I directly connect the physical servers to each other via high speed mellanox controllers (and setup network routing appropriately)?

I think I'm asking my question poorly... I apologize.

[rennerstefan]

Hi

Your second statement is right.
Data will travel through the veeam proxy(s) on source and Destination side (see https://helpcenter.veeam.com/docs/backu ... ml?ver=110).

With that there is not only no benefit in a direct connection between the hosts, it will not even work.

Thanks

[Novox]

So, I found this link where someone in the VMWare forums discusses doing exactly this for Veeam.

https://communities.vmware.com/t5/ESXi- ... d-p/915927

Can someone distill what the suggested solution is?

I have 25Gbs Mellanox controllers in each of my VMWare physical hosts and would really like the data for replication tasks to travel over these dedicated lines (to be, theoretically, 25 times faster).

[PetrM]

Hello,

The source Data Mover is running on the source proxy and reads data from production storage using one of the available transport modes, after that data is sent to the target Data Mover which writes data to the target in transport mode available for the target proxy. I think it makes sense to deploy a HotAdd proxy machine on each host and to configure the preferred network to route backup traffic over the dedicated channel from the source Data Mover to the target one.

Thanks!

[Novox]

ok, so deploy a "VMWare backup proxy" to a VM guest on each physical VMWare host, changing Transport Mode to "Virtual Appliance"? Set the preferred network(s) to my crossover direct-connect networks. (Each VMWare backup proxy guest VM will need an IP on the crossover direct-connect network).

Then, in each replication task, under data transfer, instead of "automatic selection", choose the appropriate Source Proxy (the VMWare backup proxy located on the source physical host) and Target proxy (the VMWare backup proxy located on the target physical host).

When this is done, the source datastore is hot-added to the source VMWare backup proxy, and the target datastore is hot-added to the target VMWare backup proxy. When the Data Mover starts, it will refer to the preferred network (which is my 25Gbs direct connect crossover network).

In this manner, rather than the data moving over my 1Gbs using "Network" transport (nbd), the data will move over my 25Gbps direct connections using Hot Add and this should be MUCH faster?

Does this sound correct?

[PetrM]

Yes, but I suggest to start by trying this approach using a test job in order to be sure that everything works as expected.

Thanks!

[Novox]

Hi @PetrM, I just read this:

If you back up proxies that use the Virtual appliance (Hot-Add) mode to process VM data, the change block tracking mechanism (CBT) will be disabled. For more information on CBT, see Changed Block Tracking.

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Does this mean that the solution I want to implement won't support CBT? If so, won't I lose MUCH of the performance benefit of CBT just to support a hot-add proxy?

I then found this:

If you back up a VM to which the backup proxy role is assigned and that uses Virtual appliance (HotAdd) transport mode, CBT for this VM is disabled and cannot be enabled.

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

In my scenario, I'll be creating VMWare Linux Guests whose only purpose in life is to act as Veeam proxies. The above seems to indicate that CBT would only be disabled for any VM Guest which has the hot-add virtual appliance transport mode. Since I'll never backup these linux backup proxies, perhaps this is a non-issue for my scenario?

[PetrM]

Hello,

Exactly, it's related to HotAdd proxy servers only. If you don't process proxies, then it has nothing to do with your scenario.

Thanks!

[Novox]

I've got the test environment in place.

When I add the preferred network for data to traverse my VMWare host direct connection, the data mover speed greatly increases, but everything else (i.e. job setup and control traffic) greatly decreases.

i.e. over slower production network, "Getting VM info from vSphere" takes ~15 seconds. Whole job takes 5:40. Transfer speed is 113 MB/s (aka, 1GBps).


Over preferred network, "Getting VM info from vSphere" takes ~2 minutes. Whole job takes 14:28. Transfer speed is 337 MB/s (aka, >1GBps).


If I add my slower production network to the bottom of my "preferred networks" list, the job only uses the production network (fast job setup/control, slower data movement).

IMHO, with only the preferred network defined it seems like when it's time to actually move data, it uses the preferred network. For job setup/control traffic, it's as if Veeam is timing out and falling back to the slower production network (which adds time to everything except data movement).

If true, Is there a registry setting or something to either indicate where control traffic should go OR to decrease Veeam's network timeout?

[Novox]

can't seem to find the edit button...

first image, i.e. over slower production network, "Getting VM info from vSphere" takes ~15 seconds. Whole job takes 5:40. Transfer speed is 113 MB/s (aka, 1GBps).



second image, Over preferred network, "Getting VM info from vSphere" takes ~2 minutes. Whole job takes 14:28. Transfer speed is 337 MB/s (aka, >1GBps).

[PetrM]

Hello,

I'm a bit surprised to see such a result as management traffic has nothing to do with the preferred network. I assume you didn't change a location of the backup server itself? Also, it's not clear what else besides "Getting VM into from vSphere" takes time as the total difference is 9 minutes. I don't manage to understand it from just a couple of screenshots above, I believe it's worth asking our support engineers to explain the situation based on infrastructure and debug logs analysis. Can you please open a support request and share a support case ID with us?

Thanks!

[Novox]

Thank you, @PetrM. I ended up opening a ticket with support. I have a guess as to what's going on, but not all VBR components are on the preferred networks (remember, they're just 2 host networks (/30) with fast direct connections between VMWare physical hosts).

So I think VBR is making a "best effort" to use my preferred networks (when it can, only data mover traffic) and failing back to any other network to communicate VBR infrastructure traffic which is adding time.

[PetrM]

Hello,

It's expected that management traffic ignores the preferred network setting, anyway let's see what our support team finds out: please don't forget to share a support case ID with us.

Thanks!

[Novox]

Support Case #05235591, however, it doesn't seem that management traffic is ignoring the preferred networks (until after what seems to be very lengthy timeouts for management tasks).