Backup Copy - Sync Now has 24 hour copy interval expiration?

[Eamonn]

Case 00736007

We have Veeam installed throughout our facilities and, for the most part, it functions very well

At present we are attempting to use B&R 8.0.0.917 Enterprise Plus to get an off-site backup copy of our HQ data (approx 3TB) to a remote facility approx. 30 miles distant using the Backup Copy function in conjunction with the WAN Accelerator. Intersite connectivity for the job is via VPN tunnel. The local site has 50 Mbps synchronous Internet connectivity and the target 60/4 Mbps asynchronous.

We seeded the target repository (via USB3 disk) exactly per http://www.veeam.com/kb1856 and set the backup copy interval to a couple of weeks to ensure it did not expire during the inital sync.

When we initially mapped the job to the seed backup file on the target repository and enabled, it processed for a few minutes and then reported:-

• -All tasks have been processed
  -Waiting for new copy interval

We then clicked "Sync Now" presuming there must be other processing that needed to be performed (digest/fingerprint creation etc.), but after few minutes it reported:-

• -Waiting for new restore points

Since the original backup job had been disabled and there were no changes to be copied over, this would appear to make sense.

So, we ran a single incremental backup to the local repository and then disabled the original backup job. The backup copy job finally started processing. As you might expect, the initial sync would take quite a while due to WAN accelerator fingerprint creation and larger-than-usual incremental vib. One of the VM's within the job has a 2TB volume that takes longer than 24 hours to process during the backup copy initial sync. However, despite setting a 14 day copy interval, the backup copy initial run still cancels 24 hours after the "start at" time with a "copy interval expired" error. Also, if this problem occurs during a VM volume's initial fingerprint creation/transfer period it would appear to cause the backup copy job to ignore the seed data for the VM volume that was being processed during the next (or subsequent) retries of the initial sync.

We don't understand why this occurs - especially when we've purposely defined a long copy interval for the initial run. It almost appears to be a bug in the software. Has anyone else experienced this? If we can't workaround this issue, it would seem we have little hope of getting the large VM copied over to our DR site.

[veremin]

However, despite setting a 14 day copy interval, the backup copy initial run still cancels 24 hours after the "start at" time with a "copy interval expired" error.

Did it happen at the next day when you clicked "Sync now" button? 24 hours after you clicked "Sync now"?

[Eamonn]

Vladimir, yes - from our experience it would appear to be directly associated with the "Sync now" function. ie. "Sync now" seems to have a hard-coded 24 hour copy interval. We find "Sync now" is very handy for kick-starting a backup copy interval on demand, especially to make use of WAN bandwidth over a weekend (for a potentially long initial sync with a seeded repository) when user traffic is minimal. As a point aside, my colleague found a little trick to circumvent this issue: -

• -Allow job to run for close to 24 hours
  -Disable windows time service on B&R server
  -Set the B&R Windows server clock back (incorrectly) as required, say 18 hours
  -Monitor B&R console and allow job to run to completion
  -Restart windows time service and allow B&R server to re-sync it's clock with a DC

This allowed us to get the initial backup copy sync of our large 2TB volume completed, although (as you might expect) it does wreak havoc with the job stats - at one point we saw a processing speed of 384GB/s

[veremin]

I think the behaviour described by you looks expected and covered in our help center ("sync now" section).

[Eamonn]

Thanks for the link Vladimir - I did read through the document. The following excerpt is the example from helpcenter regarding "Sync Now": -

For example, when you first created a backup copy job, you set a daily synchronization interval with the start time at 8 AM. After that, you changed the start time to 10 AM. On the start time change, you started the manual synchronization process at 1 PM. In this case, Veeam Backup & Replication will finish the current synchronization interval — that is, the synchronization interval that was started at 8 AM — immediately at 1 PM. After that, it will start a new synchronization interval. This interval will run for 21 hours — from 1 PM of the current day until 10 AM of the next day. All subsequent synchronization intervals will be started at 10 AM every day.

So, (if I am interpreting this correctly) irrespective of the fact that our job's copy interval was set at 14 days, a manually initiated "Sync Now" will terminate the following day at the exact time when the normal scheduled backup copy interval would start? In our experience it appeared to terminate 24 hours later, ignoring the fact we had a 14 day copy interval defined.

Following on from that, what would be the correct procedure for a (seeded) initial sync? (in order to avoid digest creation interruption on the large TB volume and the job then ignoring the seed data on retry) Perhaps avoid "Sync Now" and just let the copy job run as scheduled?

[foggy]

So, (if I am interpreting this correctly) irrespective of the fact that our job's copy interval was set at 14 days, a manually initiated "Sync Now" will terminate the following day at the exact time when the normal scheduled backup copy interval would start? In our experience it appeared to terminate 24 hours later, ignoring the fact we had a 14 day copy interval defined.

Yes, after clicking Sync Now, the new interval is started that lasts until the time specified for the copy interval, copy interval duration setting is ignored during this interval.

Following on from that, what would be the correct procedure for a (seeded) initial sync? (in order to avoid digest creation interruption on the large TB volume and the job then ignoring the seed data on retry) Perhaps avoid "Sync Now" and just let the copy job run as scheduled?

Letting the job do its job is indeed the correct approach. Aside from that, is there a chance that the backup used for seeding was performed more than a couple of weeks ago?

[Eamonn]

Foggy, due to numerous failed attempts the seed got to be perhaps a week old at the most. After each failed attempt we blew away the resulting .vib, overwrote the .vbr file from the USB seed, re-mapped job and then re-ran. We would like to have just sat back and waited for it to continue it's normal copy interval - but we were never able to get the job to start without clicking "Sync Now".

Would it not be more intuitive if "Sync Now" ran for a period equivalent to the backup copy interval duration (up to 14 days in our scenario) so the initial sync would complete without fear of 24 hour cancellation and the potential ignoring of the seed data during the subsequent retry This might avoid the desperation and gnashing of teeth some people endure when trying to get this working - such as this poor chap...
http://forums.veeam.com/veeam-backup-re ... 23-30.html

[foggy]

Foggy, due to numerous failed attempts the seed got to be perhaps a week old at the most. After each failed attempt we blew away the resulting .vib, overwrote the .vbr file from the USB seed, re-mapped job and then re-ran. We would like to have just sat back and waited for it to continue it's normal copy interval - but we were never able to get the job to start without clicking "Sync Now".

May I ask you, what kind of backup chain were you using for the seed? Was it a chain created by the reverse incremental backup job, by any chance (as you're mentioning .vrb files above)?

[Eamonn]

My apologies Foggy - I should have said "overwrote the .vbm" (metafile). We do indeed run a normal incremental.

[foggy]

Ok, then I would analyze job logs for the possible reason of why it didn't start copying immediately. You can ask your support engineer to assist you in that.