Backup Copy GFS Scheduling

[jfried81]

Hi,

I'm trying to understand exactly how GFS scheme of backup copy jobs works. Let's say I have:

• Backup Job running daily at 10 pm
• Backup Copy Job running daily at 6 pm
• GFS-Scheme (BCJ) with Weeklies for Sunday, 5pm, Monthlies for First Sunday of Month

Which VM-State will my archived backups contain? The one from Friday 10pm, Saturday 10 pm or Sunday 10 pm? And why?

I think, Veeam documentation is not as precise as it could be in this Point and there are reasons (on several pages) for all of them:

• It could be the state of Friday, because this one is copied on Saturday 6 pm and Sunday 5pm is part of that Backup Copy Intervall starting on Saturday.
• It could be Saturday, as this state was copied on Sunday
• It could be Sunday, as this state was originally saved on Sunday

Thanks in advance,
JF

[DGrinev]

Hi JF,

Weekly archive will contain Saturday state.
Your backup copy job will create RP and start to move data most likely after the midnight, right after the Saturday backup job will be finished.
So the date of the file is Sunday, but a state of the VM was taken from the Saturday backup.
Since you've marked Sunday as the day for archive creation by GFS Retention, the backup copy job will create a copy of the full backup file on the Sunday with the Saturday state of VMs.

[jfried81]

Hi DGrinev,

thank you for your reply. Nevertheless, it only answers my question under certain assumptions:

• You assume that the Saturday backup Job takes more than 2 hours and finishes after midnight, right? For the sake of simplicity let's assume that it only takes 1 hour and finishes before midnight. Would this change your answer?
• You assume that my Saturday Backup copy Job at 6pm doesn't find any data and waits until the Saturday backup (starting at 10 pm) is done, right? Shouldn't it be possible that the Saturday BCJ copies the Friday state?

I will try to point out my Problem in general:

• From your answer I assume, that the GFS-properties (in particular the specified time for the weekly backups) refer solely to the time, when the backup copy Job runs and creates its Restore Points. The specified time has nothing to do with the time, when the backup was originally taken by the Backup Job. Is that right?
• How does the specified time for the weeklies (in my case: 5 pm) affect the selection of the restore point that will be marked as weekly backup? For instance, in my setting there could be two backup copies created on Sunday. A delayed one from the Saturday Backup (running after midnight as you assumed) and a regular one, running at 11 pm on Sunday right after the Backup Job from 10 pm. In this setting, which restore Point would be selected as weekly backup to be kept according to the GFS-profile?

Thanks in advance,
Jan

[DGrinev]

Hi Jan,

I'm sorry I was wrong, the weekly archive will contain Sunday state in terms of previous conditions. I will explain why by answering additional questions you've asked.

For the sake of simplicity let's assume that it only takes 1 hour and finishes before midnight. Would this change your answer?

Yes, but only in terms of my previous wrong answer.

Shouldn't it be possible that the Saturday BCJ copies the Friday state?

That's possible, but it has been "copied" in terms of the predefined interval from 6 p.m. Friday till 5:59 p.m. Saturday.

From your answer I assume, that the GFS-properties (in particular the specified time for the weekly backups) refer solely to the time, when the backup copy Job runs and creates its Restore Points.

By specifying the exact time you determine that state of the VMs in weekly restore point shouldn't be earlier than 5 p.m. of Sunday.

The specified time has nothing to do with the time, when the backup was originally taken by the Backup Job. Is that right?

Right.

In this setting, which restore Point would be selected as weekly backup to be kept according to the GFS-profile?

The restore point created on Sunday after the 5 p.m. will be marked by GFS as weekly. Thanks!

[jfried81]

Hi DGrinev,

thank you for you reply. Nevertheless, it got even more confused in some points. For the sake of clarity in the remainder of this thread, I try to give some definitions. Please feel free to correct it, if necessary.

• Restore Point (RP): A RP captures the state of one VM at a specific point in time.
• Timestamp of RP: Each RP has a Timestamp, defining the point in time, when the VM-snapshot was made by the hypervisor.
• Backup-File: A Backup-File (e.g. VBK/VIB) is created by one BJ/BCJ and may contain (as long as "per-VM mode" is disabled) the (delta) states of several VMs, each having a (slightly) different timestamp.

Note: In the following, my general setting is a daily BJ with a daily BCJ copying the backup data offsite and a GFS rentention policy for the offsite backups.

1.) My first problem is the synchronization of a daily (!) BJ with a daily (!) BCJ so that the backup is copied immediately (or with a short delay) after its creation. Note: This first question is quite similar to veeam-backup-replication-f2/start-backu ... 45821.html that has also some open questions.

Example: The BJ is scheduled for 6 pm (daily) and contains VM0-VM5. Let's say, the RP's timestamps are 6:00 pm for VM0, 6:10 pm for VM1 up to 6:50 pm for VM5 each day. The BJ is already running for several days. Now, on Saturday 6:16 pm I add a BCJ (Interval starting daily at 6:15 pm). Based on the documentation, I would expect the BCJ to copy the Saturday RP of VM0 and VM1 (as they already exist at 6:16 pm) and the Friday RP of all other VMs (as they are available and have not been copied yet) into one and the same VBK/VIB-file at the target repo. As it is a daily BCJ for a daily BJ, this system could potentially continue forever and one backup file at the target repository always contains VMs from different days. Did I get it right?

foggy suggested to let the interval of the BCJ start some minutes after the BJ. If I got it right, this is not a good idea, is it? How would you solve it? BackupCopyLookForward-Flag? Another Scheduling-option? Or does it all work well, if I create the BCJ at 1pm (copying Friday's RPs immediately so that the next BCJ interval will wait for the BJ?

2.) My second problem is the GFS retention policy for BCJs and the selection of restore points. In your documentation, you (Veeam) usually mix Backup Files and their creation day (e.g. "the Saturday VIB") with restore points ("the state of Saturday"). If my thoughts at 1.) are correct (and the "per VM mode" is not enabled), one Backup File (VBK/VIB) at the target repository could contain VMs of several days. From your last answer

By specifying the exact time you determine that state of the VMs in weekly restore point shouldn't be earlier than 5 p.m. of Sunday.

I conclude that the GFS scheme refers to the RP timestamp of a VM and not to the BCJ interval or the time when BCJ runs and creates the VBK/VIB at the target repository. Assuming one VBK (with some VIBs in the chain) contains Saturday state of VM0/1 and Friday state of VM2-5 and I scheduled the Monthlies to "first Saturday". What will Veeam do when creating the monthly?

3.) My third problem is the specification of the exact time of the weeklies in GFS scheme.

By specifying the exact time you determine that state of the VMs in weekly restore point shouldn't be earlier than 5 p.m. of Sunday.

What happens, if the BJ runs daily at 3 p.m. Then, following your words, there is no RP on Sunday after 5 p.m.. Which restore point will be then be taken for weekly backup? The next one from Monday 3pm? From my point of view this would be very scary as the monthlies and quarterlies (for instance: scheduled for Sunday) do not care about time: So we would archive the Sunday 3pm state for monthlies/quarterlies and the Monday state for weeklies. I don't think that Veeam works this way. Could it be that the exact time is only relevant if the BCJ interval is less than one day?


Thanks for your answer and sorry for my numerous questions. But I have to design a backup schedule for several independent parties/sites and I want to be sure that it will work before publishing it...

Best regards
Jan

[Deon]

I feel like I would be unable to answer all your questions precisely because they are indeed a bit difficult due to a not-so-transparent nature of Backup copy intervals.
I will try to help you with the picture.

Main answers to your initial question:
• VM state depends on the time of the task start:
---- If you have a job for 1 VM or all VMs start at once, its state will be the start of the job (snapshot creation).
---- If you have many VMs in the job, VM backup tasks may start at different times and thus you will have different VM states not directly relevant to the job start time.

• Your Weekly and Monthly GFS date/time settings fit the same copy interval, so both types of GFS fulls will be created at the same time (when they overlap, it will still be a single full file, simply marked both as Weekly and Monthly).

Diagram:
- Green filled rectangle : backup job data processing
- Brown empty rectangle: copy interval
- Brown filled rectangle : backup copy data processing



As you may see, the states of the VMs will be the same as VM states taken in the backup job run on Saturday at 10 PM.
Of course there may be considerations like: if original job backup lasts longer than 20 hours, the backup copy would deliver it only on the next day, but it's unlikely.

Hopefully this picture will help you to evaluate your own setup, if you still need some clarifications, please let me know.

TL/DR: In the end it is really impossible to 100% predict what the VM state will be in the target GFS backup. Especially if source job fails just before, or takes too long, or VM tasks start at unexpected times.

However you will not make a mistake if you treat it as:
- Backup copy job creates GFS full backups with the backup data available on every Sunday, with the last available not corrupted state of the VMs in the source backups at that time.
This evaluation will be 100% correct.