WAN Accelerator first sync after seeding / best practise

[CeeAr]

Hi,

we are evaluating backup copy jobs to a offside location. (connected via MPLS - 50 MBit)
We "seeded" all jobs to the location (http://www.veeam.com/kb1856)

Now, we are trying to do the first sync with WAN Accelerator activated. As stated in the KB-Article the first sync will take longer:

Be advised, if you are using our WAN Acceleration tool with your Backup Copy job, it will have to sync a disk catalog from the source Wan Accelerator to the target Wan Accelerator. You will notice that the job will run longer on its first Sync. Direct mode, will not experience this delay.

There is one "big" VM (6 TB - file server) and it takes hours to create the fingerprint for the first hard disk (> 2 hours) and after that B&R fully reads the first hard disk ( 1 TB / 6 hours ) and then transfers 3 GB

First question is, does it make sense to use WAN accelerator in this scenario? Creating / using cache might take longer than transferring "direct"?

Second, the job above did not manage to finish. After the second disk (again, 1 TB) the WAN cache directory runs out of space (100 GB assigned - disk full after consuming 160 GB) and the job failed. We have now an incomplete restore point.
Should i just restart the job or do i need to "re-seed" the backup files?

Third question, we need to backup about 70 VMs with 8 backup jobs. What is the best way to copy them offside? One copy job for all VMs? One to one copy jobs ?

Thank you!

[dellock6]

Hi,
hhonestly with 50Mibt bandwidth I would also try direct mode, just to be sure. There can still be gains by using wan accelerator at that speed, but they can be obtained sometimes by using multiple accelerators and working with the streams values in the wan configuration. The default 5 is pretty low for that bandwidth, try to raise it and see if performances are improving.

That said, the initial fingerprint of source virtual disks cannot be skipped, (but sounds strange because 2 hours is a lot even for such a large disk, what is the source storage?) so first job will take longer anyway (unless you use direct mode). And cache consumption with a 6TB disk involved is expected, try to increase the cache if possible.

[Shestakov]

Hi CeeAr,

Second, the job above did not manage to finish. After the second disk (again, 1 TB) the WAN cache directory runs out of space (100 GB assigned - disk full after consuming 160 GB) and the job failed. We have now an incomplete restore point.
Should i just restart the job or do i need to "re-seed" the backup files?

During next job cycle backup copy job will transfer the data it was missing on the previous run. So you just need to wait for the next run.
If your backup copy jobs cannot complete within the sync interval, then you can either extend the interval or re-configure your backup copy jobs to send less data (use jobs with less VMs added) to the repository.

Third question, we need to backup about 70 VMs with 8 backup jobs. What is the best way to copy them offside? One copy job for all VMs? One to one copy jobs ?

The best practice is to group similar VMs into the same backup job for better dedupe ratios. And for the backup copy jobs, is recommended to group VMs based on the required retention policy settings and RPOs.
Thank you.

[CeeAr]

The default 5 is pretty low for that bandwidth, try to raise it and see if performances are improving.

Thanks, i will try that on the next sync.

That said, the initial fingerprint of source virtual disks cannot be skipped,

So, this fingerprinting should only happen on the first sync?

(but sounds strange because 2 hours is a lot even for such a large disk, what is the source storage?)

Backup and target storage is a local attached SAS - RAID6. Target : 9 X 3TB Local: 15 X 3 TB. The fingerprints are created on target side, right? I'm always getting "Fingerprints are missing at source, loading them from target
". May be there is a bottleneck.

Thank you, Nikita. I'm going to extend the interval.

Cheers,
Chris

[dellock6]

- yes, only on the first run
- fingerprints are created at target side when you prepopulate cache, and the message you see is about transferring those fingerprint to source to sync the cache content in terms of digests (digests of the block stored in target cache). This is a "bottleneck" in that, before starting to effectively replicate, the two wan caches are syncronizing these info. You do not see any progress in terms on the job, but underneath there is this sync running. And if cache is pretty big, on slow links even this digest sync can time some time.

[doomi]

And if cache is pretty big, on slow links even this digest sync can time some time.

how long can this take?
creating fingerprints took almost 36h and now it's 'stuck' at "loading them from target" since more than 24h.
this is for a 700GB VMDK, which I'm trying to attach the seeded VM.

[foggy]

Depends on the link speed. Digests size can be estimated as 20GB per 1TB of source data.

[doomi]

wow!
then the job should show a percentage or something, so it's visible if the job is still running or not.
I'm suggesting this as a feature request

[foggy]

Thanks for suggestion, sounds reasonable.