GFS Retention Policy

[foggy]

Then enable GFS on the second backup copy job to have them as fulls.

[cdfosburg]

Hello I opened a case 00890281, but I am confused over the steps to take to setup Veeam for what we are looking to do. Don't get me wrong, she was very helpful, this is more of a trying to wrap my mind around it situation.

I have a backup copy job that currently runs at a 1 day interval at 12:00AM EST. I have backup jobs that run at 10:00PM EST. What my issue is revolves around my weekly backups. I have chosen to make a weekly back up at 10:00AM, however the job is not running. I don't see any weekly for 4/26 in the properties of the job.

Now, I think that if I change my copy interval to every hour, that would trigger it at 10:00AM on Sunday, but I am not sure.

Let me know what else I can provide or any details for help from the community.

Thank you,

Chris

[foggy]

Chris, please review the explanation of backup copy job weekly retention for better understanding of how it works. The thread above can also clear things up for you.

[cdfosburg]

So I think this statement helps "Weekly backups are not created in a separate task. Veeam Backup & Replication re-uses a full backup created in the regular backup cycle and propagates it to the weekly tier."

I was looking for it to do something at 10:00AM on Sunday and it would not, has the cycle would not start again until Midnight on Monday morning. I have changed my cycle to run every 1 hour. However my jobs only run at 10:00PM, I hope this is not overkill. It wont copy anything cause there is nothing new on the source, so it should not impact the business operations.

As a side note, seems silly to have an option to say when you want the weekly backup created, if it will not do it at that time as a scheduled task. But at this rate, I just want stable offsite backups.

[foggy]

Indeed, no sense in setting copy interval to less then the period of your source backup job is (there will simply be nothing to copy). Weekly backups will start appear after reaching the number of restore points specified in the regular retention, so you just need to wait.

[cdfosburg]

So I have broken my jobs into several copy jobs. I am unclear how my daily sync schedule works. I read the below in the Veeam Guide

When a new synchronization interval starts, Veeam Backup & Replication checks if a new restore point is available on the source backup repository.

What about during the interval? Here is what I am doing.

Backup job starts at 10:00PM EST. A few servers, generally small incremental etc.
Backup Copy Interval starts at 2:00AM EST. I assume it would look to see the backups that were created with the 10:00PM run. However lets assume the backup job only took 30 minutes and completed at 10:30PM.

At 10:30PM is the copy job seeing the new data to copy offsite, or will it wait till the 2:00AM for that milestone?

[veremin]

At 10:30PM is the copy job seeing the new data to copy offsite, or will it wait till the 2:00AM for that milestone?

It depends on whether a backup copy has already created a restore point within that interval (interval that started yesterday at 10:30 AM) or not.

In general, it's recommended to set start of sync interval closer to start of source backup job. In your case, 10:05 would be a good option.

Thanks.

[lennis40]

We are finding it very difficult to position GFS long term retention with some of our customers, due to the enormous amount of storage required for GFS restore points. Talking with a customer today, that is required to keep 12 weekly, 36 monthly, and 10 yearly, because of external auditing. They don't have a lot of data, but using the restore point calculator, their minimal 2TB raw would be nearly 56TBs of cloud storage. This is an extreme case, due to the requirements, but the overall fact that each GFS restore point is a full, seems to create lots of issues on the storage side. Are there any ideas in the making on changing the way GFS restore points are stored, mainly being not all fulls?

[Gostev]

Simple question:
Do they actually care to be able to recover from those restore points?

If yes, then the chance of corruption in long incremental chain is too high to seriously consider this solution.
If no, then why not just backup to /dev/null instead disk space requirement problem solved!

On a serious note, anyone with retention requirements longer than 1 year should seriously consider tape. Because if you really care to be able to recover, then those GFS restore points MUST be independent full backups. Not even mentioning that 10 years is way too long time for ever spinning media.

[lennis40]

Simple question:
Do they actually care to be able to recover from those restore points?

If yes, then the chance of corruption in long incremental chain is too high to seriously consider this solution.
If no, then why not just backup to /dev/null instead disk space requirement problem solved!

On a serious note, anyone with retention requirements longer than 1 year should seriously consider tape. Because if you really care to be able to recover, then those GFS restore points MUST be independent full backups. Not even mentioning that 10 years is way too long time for ever spinning media.

Valid point, and I Agree. We weren't considering it. This customer basically wasn't going to give us the time of day to go over our offering, because all cloud based options for him have always been much too expensive with his retention requirements. Backing up to /dev/null will basically show a successful backup, but discard anything written to it, correct? Never thought of that, but I like it! Thanks for the feedback, Gostev!

[lennis40]

How do we go about sending to /dev/null? Null is a file, so how does one add as a repository?

[veremin]

I believe it's not possible. It's more about a joke, rather than a serious proposal.

[Gostev]

Yes, obviously it was just a joke

What I meant is that if you do not really care about ability to actually recover from those backups in a few years - and disk space costs is the only thing important to you - then you might as well backup to /dev/null, just to get successful backups reported

[lennis40]

Sure thing, not a linux person, thought I'd ask.

[lennis40]

I will have to read up on my Linux humor before asking questions! The reason I asked was because it would be a great thing to use in a demo or training environment, and not have to worry about storage.

[dellock6]

The complete joke I did with Anton for /dev/null is: it's awesome for backups, just don't ask for a restore

[lennis40]

So how about an idea for a repository that could be used to offload only copy job GFS restore points to. If a customer needs weekly, monthly, yearly, the cloud repo can get quite large. If we can add in a second repository, and have only those weekly, monthly, yearly GFS restore points go there, the cost wouldn't be as high with the lower tier of storage. I assume we could use file copy to move the GFS restore point wherever, but that doesn't allow it to be removed from the cloud repository.

Meeting some of the retention requirements today are easier to do onsite, but when we are talking about getting that data offsite, people balk at the amount of data created, the costs associated, and it's not always an automated process. Just looking for ideas on offloading some of those bigger restore points to cheaper storage. Thanks.

[Gostev]

Yes, we have something like this in mind for the future versions. Thanks for confirming our thinking

[lennis40]

Sounds great, thanks!

[btmaus]

OK I have a question regarding GFS retention.

I have a backup job that is keeping 7 restore points, and when creating a backup copy job for GFS retention, I am forced to keep a minimum of 2 restore points. What is this setting for and can I not disable it or set it to 0? All I want is weekly, month and yearly as the main backup job is already keeping the daily points.

[foggy]

This setting is for retention of the regular backup copy job chain (imposed by safety reasons).

[btmaus]

Thanks foggy.

A few more questions for you on this topic:

1. Is there a way to have the backup copy job start automatically once the backup job is complete?
2. Should I have one backup copy job for all my backup jobs, or should I have one backup copy job per backup job?
3. Further to the above, does the backup copy job create one large file with whatever backup jobs have been selected?

Cheers

[foggy]

1. Schedule the sync interval to start a few minutes later than the original backup job starts. This way, backup copy job will monitor the newly created restore points and start transferring data immediately.
2. It's totally up to you (you can search forums for some considerations regarding this or that approach: dedupe, ease of management, etc.).
3. Yes.

[btmaus]

OK great, thanks for answering those foggy.

When the backup copy job kicks off, which Veeam servers are responsible for processing the backup copy job data? Does the Veeam proxies do this, or is it the Veeam Backup Server?

[foggy]

Data is processed by Veeam data movers installed on the repository servers, proxies are not involved in backup copy jobs.

[btmaus]

Data is processed by Veeam data movers installed on the repository servers, proxies are not involved in backup copy jobs.

I had a look, and my Veeam B&R server is the one doing all the work (not the proxies as you mentioned). Is it possible to move the components that do this work to a separate server?

[btmaus]

Also, how do I check on the backup chain to see where my weekly/montly/yearly backups are? At the moment I can only see the restore points specified in the backup job, and the restore points specified in the backup copy job.

[Gostev]

I had a look, and my Veeam B&R server is the one doing all the work (not the proxies as you mentioned). Is it possible to move the components that do this work to a separate server?

Sure. Just create new backup repositories and proxies on another server (not a Veeam B&R server).

Also, how do I check on the backup chain to see where my weekly/montly/yearly backups are?

Go to Backups, right-click the job and open its Properties.

[btmaus]

I see one restore point has a "R" under retention. This Saturday was supposedly to be the first weekly (normal backup job keeps 7 restore points, backup copy keeps 2 restore points, 4 weekly, 12 monthly, 7 yearly), but it doesn't seem to have created it?

[Gostev]

Feel free to open a support case to troubleshoot this.