DATA DOMAIN Mtree replica

[rudebaldo]

Hello,
I am quite new to Veeam, so I am a little puzzled about Data Domain and its native replica capabilities.
This is the environment:
One single Veeam B&R server 9.5 U2 - Headquarter
One Data Domain 2200 5.6.0.5 - DDBoost and replication license installed - Headquarter
One Data Domain 2200 5.7.3.0 - DDBoost and replication license installed - Branch office connected via MPLS

The DD2200 HQ is the destination of all Veeam backup, configured as Deduplicating storage appliance

Now my goal is to replicate the data between the two Data Domain:in case of catastrophic failure I must be able to configure a Veeam server and connect to the DD2200 located in branch office, and restore VM - OR - restore the data from remote DD in case of main DD failure.
More, the replication between the two data domains must be done considering that are located on different locations.

I have tested the Data Domain Mtree native replication, but in this post:
vmware-vsphere-f24/data-domain-mtree-re ... ca#p233492
there is this warning:

But please note that we do not support or recommend storage-based replication as means to copy backups in any case, you should be using Backup Copy jobs instead.

Basing on this, I assume that this is to be done:
1) configure a Mtree on the remote DD2200
2) configure it as additional repository in the Veeam B&R server,
3) configure a backup copy job
4) configure the backup job with flag on "Secondary destinations for this job", and use the backup copy job

Is this correct, or is there any other possible configuration?
If this is the only config suggested, will the backup copy job have the same performances - in terms of data transferred and overhead - as the native Mtree replication?
Why the storage-based is not supported / recommended?

Thanks in advance

[Gostev]

Storage-based replication is not recommended because it has no content awareness and as such will replicate corrupted backups just as well as it replicates good backups.

[DeadEyedJacks]

Some Pros and Cons
With Mtree replication the offsite replica is a read-only mirror of the source.
With Veeam Copy you have an independent chain of retention points, which can have different granularity

If using Mtree replication you will need to rescan the target repo to sync its contents
With Veeam Copy you should ideally have a Veeam proxy acting as a gateway server to Data Domain at each site and utilise DDBoost.

For context
We initially setup our estate using Mtree replication, but have since changed over to Veeam Copy functionality using DDBoost.
Due to the content awareness within Veeam Backup / Enterprise Manager and the flexibility to have different GFS retention for the offsite copies.

[ferrus]

What is the performance difference between your DD MTree Replication, and Veeam Copy -> DD ?
Was is over a WAN link?

I created the original thread on the Veeam's DD MTree performance issues - veeam-backup-replication-f2/datadomain- ... 45-30.html, and tried a test Backup Copy as an alternative. It didn't seem a viable alternative at the time.

Would like to hear people's experiences migrating from DD MTree repl to Veeam BCJ. Will it increase DD capacity usage much with fresh GFS points, for instance?

[DeadEyedJacks]

Hi,
Don't have hard numbers for performance difference, as the approach differs.

We've found Veeam direct transfers between DDBoost enabled Gateway servers to be fast and efficient, with better visibility of progress and integration into Veeam console and enterprise manager.
With mtree replication the completion time seems like a London bus timetable and additional steps are needed to rescan mirrors for Veeam awareness.

(Veeam WAN acceleration, both required too much disk space for hashing and caching and took too long to compute those, so direct transfers were better for us.)
(We have six small to midrange Data Domains, total of ten Petabytes of backups sets, stored within one Petabyte of Dedupe storage for 750 VMs across six locations. Transfer links are all 1Gbps MPLS)

I wouldn't of thought the replication method will change the DD capacity usage substantially once the cleaning cycle has run.
However, with Mtree replication we did have issues with the target tree being substantially larger than the source during seeding and syncing, until cleaning runs.
So temporarily putting a squeeze on system capacity.

For us having less offsite copies, with a higher granularity was the key to changing.

[ferrus]

I might give it another try.

I was concerned because our BCJs use GFS, reading data from the source - so do Active Fulls every weekend.
The thought of this going over the WAN is scary; but if the DDboost reduces this traffic - maybe it isn't so bad.

The Veeam servers at our primary site, were each spec'd with a 960GB SSD - for WAN acceleration. When we came to install however, the documentation advised that WAN acceleration was redundant for all but the slowest comms lines.

This post from Gostev seems to suggest that for BCJ's to provide DD WAN replication, they become useful again ...

It is important to keep in mind that supportability for DD Boost over WAN is defined by EMC, not Veeam.

Here's the official WAN link requirements from EMC:

Code: Select all

Boost over WAN – up to supported tolerances

Line Speed - 10 Gb/s
RRT (ms) – 15
Packet Loss % - 0.05%

Requests for anything greater that 15ms requires RPQ.

I doubt there are many companies out there who can boast 10 Gb/s WAN link, so getting the data over WAN using Veeam's built-in WAN accelerators still remains the only option for most.

I can gain the extra working capacity on the second DD - by reducing the GFS amount as you suggest.
DD improvements have gone quiet for v9.5u3/v10 - so I'll see about migrating to BCJs.

[DeadEyedJacks]

We don't do the active full copies, just incrementals and synthetics.
We are moving to 10Gbps WAN links, have one live now and rest to do in 2018

So just by way of example;
We have an Onsite Veeam Backup Copy job with took 33 hours to move 767GB within the same DD4200 to create a new weekly restore point.
Offsite Veeam Backup Copy job over 10Gbps WAN to another DD4200 using Physical Veeam Gateway servers at both ends took 33 hours also.
(This is weekly incremental copy from daily backups with Synthetic fulls being created by Veeam using DDBoost on the Data Domains.)

Over the same 10Gbps link, 3TB of Mtree change can take 12 hours to be processed. So mtree throughput seems less to do with link speed and more to do with DD replication processes.

Also the systems are busy pretty much busy 24x7 doing backups, copies, replicas,
I get the impression that DDBoost is a front-end activity which gets priority other Mtree replication as a system activity, if that makes sense.

[rudebaldo]

With mtree replication the completion time seems like a London bus timetable and additional steps are needed to rescan mirrors for Veeam awareness.

In my tests I have configured a Mtree replication, then I've connected the remote DD as further deduplication storage. The remote Mtree was presented to Veeam, but even with a rescan no data were visible inside.
There is no Veeam proxy between the DD and Veeam itself, if it can be useful.

[rudebaldo]

Storage-based replication is not recommended because it has no content awareness and as such will replicate corrupted backups just as well as it replicates good backups.

Thanks