Backup Copy Intervals

[scottf99]

Hi,

Just upgraded to V8 and changed to Forever Incrementals. Had the same "issue" with V7 and Reverse Incrementals so not really relevant.

Anyway..... I don't understand Backup Copy intervals/Timing. I have read all the docs and apologise for my lack of understanding.

I backup a dozen Servers with one job to a local repository at 9pm each night and it finished at about 9:30. All good here
I have a Backup Copy job that copies this offsite. It is set to run "Any time (continuously)" and "Copy every 1 Day starting at 12am"

Last night my Backup job went as described above. Started on 18th Nov 9pm and finished 18th Nov 9:34pm.
The Backup Job started 19th Nov 4:30AM and only processed one Server. The rest are "Pending - latest restore point is already copied"

Any ideas?
Cheers
Scott

[Gostev]

Hi, Scott. Try switching Backup Copy jobs to the forward lookup mode (new in v8, see Backup Copy section of What's New document for the registry value), and let me know if you like this behavior better. I am considering making it a default in the next release, but want to hear some feedback first. Thanks!

[scottf99]

Thanks Gostev. Ill look into this and get back to you.

Is this relevant to my issue?

[scottf99]

Also can you please give me some more info on the BackupCopyLookForward method? I cant find anything except the "whats new" doc and that doesn't really explain the difference.

Cheers
Scott

[tsightler]

I backup a dozen Servers with one job to a local repository at 9pm each night and it finished at about 9:30. All good here
I have a Backup Copy job that copies this offsite. It is set to run "Any time (continuously)" and "Copy every 1 Day starting at 12am"

The registry key to do "look forward" would probably work, but I think it's only a partial solution since you'll only have the time from when the job ends until 12AM for the job to finish. Ideally you'd want to set the interval time correctly as well.

The "interval" in the Backup Copy job should be set to coincide with the time that your backup window starts. In other words, since your backup jobs start at 9PM, you want to set the interval to also be 9PM. That will tell the Backup Copy to start it's next 24 hour interval at 9PM and start looking for new points, then when your jobs ends it will immediately start copying. If you some reason you want the copy to wait until 12AM to start, you can set a "blackout window" to disallow transfers form 9PM until 12am. That way the Backup Copy will still start looking for new backups at 9PM but won't actually start transferring any data it finds until 12AM.

[scottf99]

Thanks Tom. I understand your reply but still trying to grasp the concept of "Intervals". I just want a copy of my backups to go offsite as soon as they are created regardless of date or time. Does this fit in with how you have designed Backup Copy?
Also still keen to get more tech info on BackupCopyLookForward

Cheers
Scott

[Gostev]

Yes, it does.

By default, when starting new copy interval, Backup Copy job will immediately look for the most recent backup of each VM that it has to process, and as long as this restore point has not yet been copied during the previous copy interval, Backup Copy job will start copying it.

With the BackupCopyLookForward modifier applied, Backup Copy job will instead wait for the new restore points to appear, and only copy those, as they appear. I know this is probably largely the same explanation I have put into the What's New, but I don't really know what other "tech info" on BackupCopyLookForward are you looking for, because this really is pretty simplistic stuff.

[tsightler]

Sure, it fits, but Backup Copy jobs are very flexible to meet other scenarios as well and a single Backup Copy can also monitor mulitple Backup Jobs so the concept of the "Interval" is to define two things, how often it should be looking for new points, and for a daily interval, what time it should start looking for points. If you set an interval for 1 day, that's telling the Backup Copy job that once a day, starting at the selected time, it should begin looking for new restore points and, as soon as it finds one, copy it. However, once a single point is copied, another point for that VM will not be copied until the next interval starts.

The idea behind this is that you may run backups locally more often (for example hourly), but you may only want to copy offsite once a day or weekly, thus you can set the backup copy "interval" independently of the schedule of the backup jobs it is monitoring. You can think of it almost like setting an SLA.

But in your case it sounds pretty simple. It sounds like you are running backups once a day starting at 9PM and you want them copied offsite as soon as they're done, so you set the backup copy for 1 day (same as the frequency you are running local backups and you want offsite copies), and the start time for the interval to 9PM (when to start looking for new points to copy). The backup copy will start a new interval at 9PM looking for points, the backup job will start at the same time, and as soon as the backup job completes the backup copy will start pushing that point offsite.

[scottf99]

Thank you Tom, that makes it easier to understand. When one is trying to do something at the very bottom end of what the software can accommodate then it can be as confusing as doing the really tricky stuff.

Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?

[foggy]

Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?

Not exactly. With the key specified, it will not copy the available existing restore point that was not copied previously, but will wait for the new one to be created, instead.

[VladV]

If you some reason you want the copy to wait until 12AM to start, you can set a "blackout window" to disallow transfers form 9PM until 12am. That way the Backup Copy will still start looking for new backups at 9PM but won't actually start transferring any data it finds until 12AM.

From our testing it seems that the Backup Copy Job starts looking for new restore points only when the coming out of the blackout window if the new interval start time is in the restricted transfer period.

[tsightler]

From our testing it seems that the Backup Copy Job starts looking for new restore points only when the coming out of the blackout window if the new interval start time is in the restricted transfer period.

I'm not sure I follow. The interval defines when the system is looking for points, the blackout window defines when data is allowed to be transferred. I have this exact setup both at clients and in my lab. In the lab setup backups start at 1AM, and the copy interval also starts at 1AM. When the backups complete the copy job finds the point but instead of the "waiting for restore points" message the message changes to "waiting for network transfer window" (or something like that, don't remember the exact wording) then, at 5AM when the blackout window end, the data transfer actually starts. The blackout window should not have any impact on when the job is looking for points, only when data is allowed to be transferred.

There is some difference in behavior based on how you select your backups. For example if you select by repository, rather than by job the behavior is slightly different, but I always recommend selecting by job because that behavior seems predictable to me and I can't find a lot of reasons not to do that. Are you by chance selecting "From backups" or "From infrastructure" instead? I'll try to test that scenario if so.

[VladV]

OK, I was translating blackout as is, meaning a period when you cannot send any data.

Taking your setup for example, you have a newly created restore point at 1AM. The backup copy job also starts at 1AM but is restricted to send any data until 5AM. From our tests we came to the conclusion that, if creating another restore point between 1AM and 5AM, the backup copy job will copy that restore point and not the 1AM one, making the selection when it is allowed to send data and not when the copy interval starts.

Our setup was made only for testing to see if it was viable to migrate to backup copy jobs instead of a weekly robocopy script that copies the vbks at the end of the week. We also targeted the backup job as source and a repository defined as rotating media.

[Gostev]

Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?

Normal Backup Copy job will do the check immediately upon starting to see if any uncopied restore points exist. If they do, Backup Copy job will start copying those immediately. Otherwise, if the latest existing restore point was already copied during the previous interval, Backup Copy job will keep waiting for the new restore point to appear for the duration of the interval, and will start copying data as soon as the new restore point appears.

With the BackupCopyLookForward modifier applied, Backup Copy will ignore any restore points already present at the time when interval starts (even if they were not copied), and will wait for the new restore point point to appear. So basically, it drops the first half of the previous paragraph, making the Backup Copy job skip that initial check for existing uncopied restore points at the interval start.

[blodsbror]

Support case #01017915

Hi,

I have multiple jobs (hyper-v and vmware) backing up on a daily basis, and two BCJ´s (1 for Hyper-v, 1 for Vmware). All of the daily jobs start at staggered intervals from 16:30 onwards, and all generally complete by about 10am the next morning. The BCJ´s are set to copy every 1 day, with a starting time of 10am and a schedule of ´any time´. All regular jobs are current set to use Forward Incremental.

I´ve been reading as much as I can, however I still don´t understand if this is the optimal timing for the BCJ´s to start. Support mentioned that the BCJ´s can be started at 16:30 as well, and "the newly created RP will be processed during the next backup copy interval".

Why would the data in the current backup jobs, be copied during the NEXT BCJ interval - and not the current interval ?. I really want to understand this correctly, instead of just doing as support suggests. As I´m assuming I could also start the BCJ´s at other times, and it would be just a optimal ?

Thanks!

[veremin]

Take a look at the answer provided by Tom; should clarify the interval concept for you.

As to backup copy job start time, you can set it five minutes or so after backup jobs start - 16-35.

Thanks.

[bungle]

Sorry still don't understand

I have a number of backup jobs daily incremental Mon-Fri, with full Saturday

I would like the backup copy job to only copy the fulls (created Saturday night) to the target, I had the copy every interval set to 1 day and schedule set to anytime, I have now changed copy every interval to 7 days and schedule to enable Sunday 6am - midnight every other time is disabled.

Will this do want I am trying to achieve?

[foggy]

With these settings backup copy will create a restore point each Sunday. Please keep in mind that it doesn't copy the backup file as a whole, but synthetically creates restore point corresponding to the latest VM state available in the source repository.

[bungle]

Thanks all I want to see on the target is a number of full backups and no incremental.

Backup Copy Target is an HPE StoreOnce and the requirement is long term retention of Weekly, Monthly and Yearly fulls I have ticked the box "Keep the following restore points for archival purposes" don't want to see any incremental on there, fast restores provided by DAS.

[foggy]

There will always be at least one increment in the regular backup copy chain though, since the lowest retention you can set is 2. All GFS (archival) restore points you've configured will be fulls.

[bungle]

Okay thanks for explaining

[veremin]

Thanks all I want to see on the target is a number of full backups and no incremental.

It will be possible only if you set sync interval to 7 days, enable weekly GFS as well as "read the entire restore point" option. But since you're using daily sync interval, there will be always at least one increment (as Foggy's pointed out). Thanks.

[Tecbil]

I've just updated the B&R server from Windows 10 1809 to Windows 10 1903 and since then the "backupcopylookforward" setting is not working anymore, it seems it just gets ignored. Is this a known issue or am I the only one?

[foggy]

Hi Fabio, this doesn't depend on the Windows version, so I'd check if the value is still configured and, if everything seems fine, contact support for investigation.