Backup Copy Intervals

Availability for the Always-On Enterprise

Backup Copy Intervals

Veeam Logoby scottf99 » Tue Nov 18, 2014 11:07 pm

Hi,

Just upgraded to V8 and changed to Forever Incrementals. Had the same "issue" with V7 and Reverse Incrementals so not really relevant.

Anyway..... I don't understand Backup Copy intervals/Timing. I have read all the docs and apologise for my lack of understanding.

I backup a dozen Servers with one job to a local repository at 9pm each night and it finished at about 9:30. All good here
I have a Backup Copy job that copies this offsite. It is set to run "Any time (continuously)" and "Copy every 1 Day starting at 12am"

Last night my Backup job went as described above. Started on 18th Nov 9pm and finished 18th Nov 9:34pm.
The Backup Job started 19th Nov 4:30AM and only processed one Server. The rest are "Pending - latest restore point is already copied"

Any ideas?
Cheers
Scott
scottf99
Influencer
 
Posts: 19
Liked: never
Joined: Mon Jul 29, 2013 2:13 am
Full Name: Scott

Re: Backup Copy Intervals

Veeam Logoby Gostev » Tue Nov 18, 2014 11:52 pm

Hi, Scott. Try switching Backup Copy jobs to the forward lookup mode (new in v8, see Backup Copy section of What's New document for the registry value), and let me know if you like this behavior better. I am considering making it a default in the next release, but want to hear some feedback first. Thanks!
Gostev
Veeam Software
 
Posts: 21386
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Backup Copy Intervals

Veeam Logoby scottf99 » Wed Nov 19, 2014 12:18 am

Thanks Gostev. Ill look into this and get back to you.

Is this relevant to my issue?
scottf99
Influencer
 
Posts: 19
Liked: never
Joined: Mon Jul 29, 2013 2:13 am
Full Name: Scott

Re: Backup Copy Intervals

Veeam Logoby scottf99 » Wed Nov 19, 2014 12:33 am

Also can you please give me some more info on the BackupCopyLookForward method? I cant find anything except the "whats new" doc and that doesn't really explain the difference.

Cheers
Scott
scottf99
Influencer
 
Posts: 19
Liked: never
Joined: Mon Jul 29, 2013 2:13 am
Full Name: Scott

Re: Backup Copy Intervals

Veeam Logoby tsightler » Wed Nov 19, 2014 12:35 am

scottf99 wrote:I backup a dozen Servers with one job to a local repository at 9pm each night and it finished at about 9:30. All good here
I have a Backup Copy job that copies this offsite. It is set to run "Any time (continuously)" and "Copy every 1 Day starting at 12am"

The registry key to do "look forward" would probably work, but I think it's only a partial solution since you'll only have the time from when the job ends until 12AM for the job to finish. Ideally you'd want to set the interval time correctly as well.

The "interval" in the Backup Copy job should be set to coincide with the time that your backup window starts. In other words, since your backup jobs start at 9PM, you want to set the interval to also be 9PM. That will tell the Backup Copy to start it's next 24 hour interval at 9PM and start looking for new points, then when your jobs ends it will immediately start copying. If you some reason you want the copy to wait until 12AM to start, you can set a "blackout window" to disallow transfers form 9PM until 12am. That way the Backup Copy will still start looking for new backups at 9PM but won't actually start transferring any data it finds until 12AM.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Backup Copy Intervals

Veeam Logoby scottf99 » Wed Nov 19, 2014 12:45 am

Thanks Tom. I understand your reply but still trying to grasp the concept of "Intervals". I just want a copy of my backups to go offsite as soon as they are created regardless of date or time. Does this fit in with how you have designed Backup Copy?
Also still keen to get more tech info on BackupCopyLookForward

Cheers
Scott
scottf99
Influencer
 
Posts: 19
Liked: never
Joined: Mon Jul 29, 2013 2:13 am
Full Name: Scott

Re: Backup Copy Intervals

Veeam Logoby Gostev » Wed Nov 19, 2014 12:57 am

Yes, it does.

By default, when starting new copy interval, Backup Copy job will immediately look for the most recent backup of each VM that it has to process, and as long as this restore point has not yet been copied during the previous copy interval, Backup Copy job will start copying it.

With the BackupCopyLookForward modifier applied, Backup Copy job will instead wait for the new restore points to appear, and only copy those, as they appear. I know this is probably largely the same explanation I have put into the What's New, but I don't really know what other "tech info" on BackupCopyLookForward are you looking for, because this really is pretty simplistic stuff.
Gostev
Veeam Software
 
Posts: 21386
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Backup Copy Intervals

Veeam Logoby tsightler » Wed Nov 19, 2014 1:14 am

Sure, it fits, but Backup Copy jobs are very flexible to meet other scenarios as well and a single Backup Copy can also monitor mulitple Backup Jobs so the concept of the "Interval" is to define two things, how often it should be looking for new points, and for a daily interval, what time it should start looking for points. If you set an interval for 1 day, that's telling the Backup Copy job that once a day, starting at the selected time, it should begin looking for new restore points and, as soon as it finds one, copy it. However, once a single point is copied, another point for that VM will not be copied until the next interval starts.

The idea behind this is that you may run backups locally more often (for example hourly), but you may only want to copy offsite once a day or weekly, thus you can set the backup copy "interval" independently of the schedule of the backup jobs it is monitoring. You can think of it almost like setting an SLA.

But in your case it sounds pretty simple. It sounds like you are running backups once a day starting at 9PM and you want them copied offsite as soon as they're done, so you set the backup copy for 1 day (same as the frequency you are running local backups and you want offsite copies), and the start time for the interval to 9PM (when to start looking for new points to copy). The backup copy will start a new interval at 9PM looking for points, the backup job will start at the same time, and as soon as the backup job completes the backup copy will start pushing that point offsite.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Backup Copy Intervals

Veeam Logoby scottf99 » Wed Nov 19, 2014 1:34 am

Thank you Tom, that makes it easier to understand. When one is trying to do something at the very bottom end of what the software can accommodate then it can be as confusing as doing the really tricky stuff.

Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?
scottf99
Influencer
 
Posts: 19
Liked: never
Joined: Mon Jul 29, 2013 2:13 am
Full Name: Scott

Re: Backup Copy Intervals

Veeam Logoby foggy » Wed Nov 19, 2014 11:44 am

scottf99 wrote:Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?

Not exactly. With the key specified, it will not copy the available existing restore point that was not copied previously, but will wait for the new one to be created, instead.
foggy
Veeam Software
 
Posts: 14718
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backup Copy Intervals

Veeam Logoby VladV » Wed Nov 19, 2014 12:18 pm

tsightler wrote:If you some reason you want the copy to wait until 12AM to start, you can set a "blackout window" to disallow transfers form 9PM until 12am. That way the Backup Copy will still start looking for new backups at 9PM but won't actually start transferring any data it finds until 12AM.

From our testing it seems that the Backup Copy Job starts looking for new restore points only when the coming out of the blackout window if the new interval start time is in the restricted transfer period.
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Backup Copy Intervals

Veeam Logoby tsightler » Wed Nov 19, 2014 2:33 pm

VladV wrote:
tsightler wrote:If you some reason you want the copy to wait until 12AM to start, you can set a "blackout window" to disallow transfers form 9PM until 12am. That way the Backup Copy will still start looking for new backups at 9PM but won't actually start transferring any data it finds until 12AM.

From our testing it seems that the Backup Copy Job starts looking for new restore points only when the coming out of the blackout window if the new interval start time is in the restricted transfer period.

I'm not sure I follow. The interval defines when the system is looking for points, the blackout window defines when data is allowed to be transferred. I have this exact setup both at clients and in my lab. In the lab setup backups start at 1AM, and the copy interval also starts at 1AM. When the backups complete the copy job finds the point but instead of the "waiting for restore points" message the message changes to "waiting for network transfer window" (or something like that, don't remember the exact wording) then, at 5AM when the blackout window end, the data transfer actually starts. The blackout window should not have any impact on when the job is looking for points, only when data is allowed to be transferred.

There is some difference in behavior based on how you select your backups. For example if you select by repository, rather than by job the behavior is slightly different, but I always recommend selecting by job because that behavior seems predictable to me and I can't find a lot of reasons not to do that. Are you by chance selecting "From backups" or "From infrastructure" instead? I'll try to test that scenario if so.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Backup Copy Intervals

Veeam Logoby VladV » Wed Nov 19, 2014 2:41 pm

OK, I was translating blackout as is, meaning a period when you cannot send any data.

Taking your setup for example, you have a newly created restore point at 1AM. The backup copy job also starts at 1AM but is restricted to send any data until 5AM. From our tests we came to the conclusion that, if creating another restore point between 1AM and 5AM, the backup copy job will copy that restore point and not the 1AM one, making the selection when it is allowed to send data and not when the copy interval starts.

Our setup was made only for testing to see if it was viable to migrate to backup copy jobs instead of a weekly robocopy script that copies the vbks at the end of the week. We also targeted the backup job as source and a repository defined as rotating media.
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Backup Copy Intervals

Veeam Logoby Gostev » Wed Nov 19, 2014 7:49 pm

scottf99 wrote:Gostev, so the normal Backup Copy checks once per interval where with the BackupCopyLookForward modifier applied it looks contstantly? Is that a simple explanation or am I still missing something?

Normal Backup Copy job will do the check immediately upon starting to see if any uncopied restore points exist. If they do, Backup Copy job will start copying those immediately. Otherwise, if the latest existing restore point was already copied during the previous interval, Backup Copy job will keep waiting for the new restore point to appear for the duration of the interval, and will start copying data as soon as the new restore point appears.

With the BackupCopyLookForward modifier applied, Backup Copy will ignore any restore points already present at the time when interval starts (even if they were not copied), and will wait for the new restore point point to appear. So basically, it drops the first half of the previous paragraph, making the Backup Copy job skip that initial check for existing uncopied restore points at the interval start.
Gostev
Veeam Software
 
Posts: 21386
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

[MERGED] : Forward incremental + Backup copy job interval

Veeam Logoby blodsbror » Mon Sep 07, 2015 5:17 pm

Support case #01017915

Hi,

I have multiple jobs (hyper-v and vmware) backing up on a daily basis, and two BCJ´s (1 for Hyper-v, 1 for Vmware). All of the daily jobs start at staggered intervals from 16:30 onwards, and all generally complete by about 10am the next morning. The BCJ´s are set to copy every 1 day, with a starting time of 10am and a schedule of ´any time´. All regular jobs are current set to use Forward Incremental.

I´ve been reading as much as I can, however I still don´t understand if this is the optimal timing for the BCJ´s to start. Support mentioned that the BCJ´s can be started at 16:30 as well, and "the newly created RP will be processed during the next backup copy interval".

Why would the data in the current backup jobs, be copied during the NEXT BCJ interval - and not the current interval ?. I really want to understand this correctly, instead of just doing as support suggests. As I´m assuming I could also start the BCJ´s at other times, and it would be just a optimal ?

Thanks!
blodsbror
Enthusiast
 
Posts: 49
Liked: 1 time
Joined: Wed Apr 02, 2014 7:40 pm
Full Name: Evan Williams

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Majestic-12 [Bot], quazer and 53 guests