Granular backup schedule

[ChillyHellion]

Hello, and thank you for your help. I'm new to Veeam and have a question about scheduling.

I would like to set up a schedule that keeps the following recovery points:

* nightly recovery points for a week
* weekly recovery points for a month
* monthly recovery points for a year

My goal is to have recovery points that extend back a year's time, but provide increasing recovery point "resolution" for more recent recovery points, with longer gaps between older recovery points.

Below is my schedule as implemented, but I'm looking for any best practice advice on how to set up this kind of schedule. I found that I had to set up three different backup jobs, which I'm not sure is the best way to achieve this result:

* Nightly Incremental - Executes 7pm - Monday through Thursday - Retain 5 recovery points
* Weekly Synthetic Full - Executes 7pm - Friday - Retain 5 recovery points
* Monthly Full Backup - Executes 7pm - Last Saturday of every month - Retain 13 recovery points


I apologize if this is a double post. It looks like I posted this under an existing thread before, which was not my intention.
Thank you.

[PetrM]

Hello Kyle,

I think you could configure just one primary backup job and a backup copy job with GFS retention instead of using 3 different backup jobs.
By the way, GFS retention for primary backup jobs will be available in the version 10.

Thanks!

[ChillyHellion]

Hello PetrM, and thank you very much for your quick and straightforward solution! I haven't heard of GFS retention before, but it looks like exactly the thing I'm looking for. Is there any way to implement GFS retention in Veeam 9.5, or is that exclusively for Veeam 10? (I'm not sure what was meant by "primary backup jobs" in this context).

If it's only in Veeam 10, would it be advisable to stick with my three-job schedule listed above and switch to GFS retention when it's available? Thank you very much for your help today, your time and assistance are very much appreciated.

[ChillyHellion]

Hi PetrM, I read through the documentation regarding backup copy jobs and I think I understand your proposed solution. I would need to set up a single backup job that runs nightly with a week's worth of retention, then set up a secondary copy job that copies the full backup recovery points from the single backup job, and retains those copies on a GFS schedule. I would need to maintain this setup only until Veeam 10 releases, at which point I can schedule the primary copy job with GFS retention (and no need for a secondary backup copy job).

I'll give this a try, thank you so much for your help today, and have a happy new year!

[PetrM]

I'm glad to hear that the information provided was helpful. However I'd like to clarify just one detail:

then set up a secondary copy job that copies the full backup recovery points

In fact, backup copy job does not transfer source backup files, it transfers actual state of VM from the whole set of source backup files.
It accumulates blocks representing actual VM state and transfer these blocks to a target where backup copy creates its own chain: full + further increments.

Have a Happy New Year!

[ChillyHellion]

I had to read it a few times, but I think that makes sense now. Does this mean that instead of simply copying over the recovery points (incremental or full), a backup copy job instead generates the full state of the VM and sends it to the backup repository to be handled as determined by the repository? And that the backup repository either creates a new full recovery point or an incremental recovery point depending on how it is configured?

If this is the case, does the backup copy have to have its own chain, or can the backup copy merge into the existing primary backup job's chain? Would I need to provide the backup copy job with its own retention plan?

This is all pretty complicated; I can see why adding support for GFS scheduling within the primary backup job is on the roadmap. It seems like a much simpler approach.

[PetrM]

backup copy job instead generates the full state of the VM and sends it to the backup repository to be handled as determined by the repository?

The session type (full/incremental) does not depend on repository, session type is defined by backup copy job settings only.
By default, backup copy transfers full VM state during its first run and all subsequent runs are incremental.
You can enable "Read the entire restore point from source instead of synthesizing it from increments" option to transfer full VM state from source periodically.

does the backup copy have to have its own chain

Yes, correct, it creates its own backup chain and has its own retention. It never performs merges with a source backup chain.

Thanks!

[Virtuollie]

You did understand quite well - a backup copy job does indeed open a new backup chain with method of forward incremental forever. If you setup GFS retention within the copy job, you will have synthetic full backups at the defined schedules. The method is not determined by the repository, it is always forward incremental, native to a backup copy job.
You are also right with the retention plan for the copy job, it is completely independent from the retention of original backup job.

[ChillyHellion]

Awesome, I think that wraps things up for me. Thank you again!

[ChillyHellion]

I'm running into an issue now where the backup copy jobs are running long, into the production hours of the next day. If I were to stick with my original plan of creating three backup jobs with different frequencies (nightly, weekly, and monthly), would all of these backup jobs save their backups in the same backup chain, or would this setup create three independent backup chains? Can I set the retention per job, or is retention determined per chain?

Thank you for your help.

[PetrM]

Hello Kyle,

Each backup job always has its own chain and retention setting.

Why do you think that backup copy requires more time to complete data transfer than 3 backup jobs in total?
Bearing in mind that backup copy does not read data from production but uses backup files on repository instead, I'd like
to clarify what is the impact on the environment because of this?

One more consideration is that the first backup copy interval of the backup copy job always produces a full backup file.
Further incremental runs need to transfer less data and obviously will have lower duration.

Thanks!

[ChillyHellion]

That makes sense, thanks PetrM! I only noticed because the backup copy job that I began on Friday was still running when I returned to the office on Monday, which is unusual for our environment - I only have seven small VMs included in my test backup schedule. I'll need to go over my backup copy settings and make sure they're correct. I don't think three backup jobs is the right solution either, because I have no need for creating three backup chains. With backup copy, I should have two chains, correct? The primary backup job and the GFS copy chain?

[ChillyHellion]

Okay, I'm stumbling through this, but have some retention questions if you don't mind. When I choose the restore points to keep, I would need to total the number of restore points in all of my periodic scheduled full backups, correct? Meaning, if I have a GFS copy job that keeps 5 weekly and 13 monthly restore points as full backups for archival purposes, I would need to specify "18" as the number of restore points to keep? Will the backup copy job that I've outlined create any incremental recovery points at all, or will it only create full backups?

I think I also see where I may have gone wrong before. I was thinking that the backup copy job executes a backup on the scheduled day if no backup exists. So I was setting my nightly backups for Mon-Thu and the copy job on Fri. Now I see that the copy job doesn't create its own recovery points under any circumstances, and my Friday copy job was just copying Thursday's copy job, being the most recent. So what I'll want to do is schedule a regular backup job for Mon-Fri, retain 6 recovery points, and schedule a backup copy job to run on Saturdays that keeps full backups according to the GFS schedule.

I also saw a note in the user guide that mentioned not to use Synthetic fulls if you have a deduplicating backup appliance as your file share. I do, but it's an EMC Data Domain that should be exempt from this consideration, per the user guide.

[ChillyHellion]

Or would I need to set the following schedule for GFS backup copies:

Type: backup copy job
Copy every 7 days
Restore points to keep: 5
keep the following restore points as full backups: monthly backup: 13

[DGrinev]

There are two retention policies for the backup copy job: simple retention policy and GFS retention policy.
The restore points to keep relates to the simple retention that doesn't affect the amount of GFS restore points. It creates a set number of restore points as a secondary backup chain.

Thanks!

[foggy]

I also saw a note in the user guide that mentioned not to use Synthetic fulls if you have a deduplicating backup appliance as your file share. I do, but it's an EMC Data Domain that should be exempt from this consideration, per the user guide.

Provided you have DD Boost enabled.

[ChillyHellion]

There are two retention policies for the backup copy job: simple retention policy and GFS retention policy.
The restore points to keep relates to the simple retention that doesn't affect the amount of GFS restore points. It creates a set number of restore points as a secondary backup chain.

Thanks!

Clear and concise, thank you so much!

[ChillyHellion]

Provided you have DD Boost enabled.

I double-checked and we don't have DD Boost licensed; thank you for providing this information!

[ChillyHellion]

Hello again. I tried to run the GFS copy job schedule that we ironed out, but I'm getting the error "Restore point is located in Backup Copy target repository and cannot be used as a source". I thought that maybe a backup copy job would be a good solution, but it's looking more and more like Veeam just doesn't have a good solution for GFS as part of the primary backup plan just yet. I may have to just wait until this feature is supported as part of the primary backup job, since we'd be losing that feature moving from Arcserve UDP.

I appreciate your help gentlemen, I think this is probably just not GFS Copy Job's intended use case, since it's likely intended to be used with a secondary backup repository.

[foggy]

Hi Kyle, right, backup copy cannot use the same backup repo both as source and target. You can create another repository on the same storage just pointing to the folder next to the source one. Thanks!

[ChillyHellion]

That may be a good workaround, thanks Foggy, I appreciate it!

I see from Veeam's dashboard that v10 (that includes primary GFS backup jobs) is coming in just three weeks, is that correct? If so, I may just hold out until then as we're still waiting on pricing from our VAR anyway.

https://go.veeam.com/v10

[veremin]

Correct. Thanks!