Comprehensive data protection for all workloads
Post Reply
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy »

Then enable GFS on the second backup copy job to have them as fulls.
cdfosburg
Influencer
Posts: 22
Liked: never
Joined: Jan 07, 2015 9:14 pm
Contact:

[MERGED] GFS Schedule Weekly

Post by cdfosburg »

Hello I opened a case 00890281, but I am confused over the steps to take to setup Veeam for what we are looking to do. Don't get me wrong, she was very helpful, this is more of a trying to wrap my mind around it situation.

I have a backup copy job that currently runs at a 1 day interval at 12:00AM EST. I have backup jobs that run at 10:00PM EST. What my issue is revolves around my weekly backups. I have chosen to make a weekly back up at 10:00AM, however the job is not running. I don't see any weekly for 4/26 in the properties of the job.

Now, I think that if I change my copy interval to every hour, that would trigger it at 10:00AM on Sunday, but I am not sure.

Let me know what else I can provide or any details for help from the community.

Thank you,

Chris
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy » 1 person likes this post

Chris, please review the explanation of backup copy job weekly retention for better understanding of how it works. The thread above can also clear things up for you.
cdfosburg
Influencer
Posts: 22
Liked: never
Joined: Jan 07, 2015 9:14 pm
Contact:

Re: GFS Retention Policy

Post by cdfosburg »

So I think this statement helps "Weekly backups are not created in a separate task. Veeam Backup & Replication re-uses a full backup created in the regular backup cycle and propagates it to the weekly tier."

I was looking for it to do something at 10:00AM on Sunday and it would not, has the cycle would not start again until Midnight on Monday morning. I have changed my cycle to run every 1 hour. However my jobs only run at 10:00PM, I hope this is not overkill. It wont copy anything cause there is nothing new on the source, so it should not impact the business operations.

As a side note, seems silly to have an option to say when you want the weekly backup created, if it will not do it at that time as a scheduled task. But at this rate, I just want stable offsite backups.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy »

Indeed, no sense in setting copy interval to less then the period of your source backup job is (there will simply be nothing to copy). Weekly backups will start appear after reaching the number of restore points specified in the regular retention, so you just need to wait.
cdfosburg
Influencer
Posts: 22
Liked: never
Joined: Jan 07, 2015 9:14 pm
Contact:

Re: GFS Retention Policy

Post by cdfosburg »

So I have broken my jobs into several copy jobs. I am unclear how my daily sync schedule works. I read the below in the Veeam Guide

When a new synchronization interval starts, Veeam Backup & Replication checks if a new restore point is available on the source backup repository.

What about during the interval? Here is what I am doing.

Backup job starts at 10:00PM EST. A few servers, generally small incremental etc.
Backup Copy Interval starts at 2:00AM EST. I assume it would look to see the backups that were created with the 10:00PM run. However lets assume the backup job only took 30 minutes and completed at 10:30PM.

At 10:30PM is the copy job seeing the new data to copy offsite, or will it wait till the 2:00AM for that milestone?
veremin
Product Manager
Posts: 20261
Liked: 2249 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS Retention Policy

Post by veremin »

At 10:30PM is the copy job seeing the new data to copy offsite, or will it wait till the 2:00AM for that milestone?
It depends on whether a backup copy has already created a restore point within that interval (interval that started yesterday at 10:30 AM) or not.

In general, it's recommended to set start of sync interval closer to start of source backup job. In your case, 10:05 would be a good option.

Thanks.
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

We are finding it very difficult to position GFS long term retention with some of our customers, due to the enormous amount of storage required for GFS restore points. Talking with a customer today, that is required to keep 12 weekly, 36 monthly, and 10 yearly, because of external auditing. They don't have a lot of data, but using the restore point calculator, their minimal 2TB raw would be nearly 56TBs of cloud storage. This is an extreme case, due to the requirements, but the overall fact that each GFS restore point is a full, seems to create lots of issues on the storage side. Are there any ideas in the making on changing the way GFS restore points are stored, mainly being not all fulls?
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS Retention Policy

Post by Gostev »

Simple question:
Do they actually care to be able to recover from those restore points?

If yes, then the chance of corruption in long incremental chain is too high to seriously consider this solution.
If no, then why not just backup to /dev/null instead ;) disk space requirement problem solved!

On a serious note, anyone with retention requirements longer than 1 year should seriously consider tape. Because if you really care to be able to recover, then those GFS restore points MUST be independent full backups. Not even mentioning that 10 years is way too long time for ever spinning media.
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

Gostev wrote:Simple question:
Do they actually care to be able to recover from those restore points?

If yes, then the chance of corruption in long incremental chain is too high to seriously consider this solution.
If no, then why not just backup to /dev/null instead ;) disk space requirement problem solved!

On a serious note, anyone with retention requirements longer than 1 year should seriously consider tape. Because if you really care to be able to recover, then those GFS restore points MUST be independent full backups. Not even mentioning that 10 years is way too long time for ever spinning media.
Valid point, and I Agree. We weren't considering it. This customer basically wasn't going to give us the time of day to go over our offering, because all cloud based options for him have always been much too expensive with his retention requirements. Backing up to /dev/null will basically show a successful backup, but discard anything written to it, correct? Never thought of that, but I like it! Thanks for the feedback, Gostev!
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

How do we go about sending to /dev/null? Null is a file, so how does one add as a repository?
veremin
Product Manager
Posts: 20261
Liked: 2249 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS Retention Policy

Post by veremin »

I believe it's not possible. It's more about a joke, rather than a serious proposal. :)
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS Retention Policy

Post by Gostev »

Yes, obviously it was just a joke :D

What I meant is that if you do not really care about ability to actually recover from those backups in a few years - and disk space costs is the only thing important to you - then you might as well backup to /dev/null, just to get successful backups reported ;)
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

Sure thing, not a linux person, thought I'd ask.
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

I will have to read up on my Linux humor before asking questions! The reason I asked was because it would be a great thing to use in a demo or training environment, and not have to worry about storage.
dellock6
Veeam Software
Posts: 6137
Liked: 1926 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: GFS Retention Policy

Post by dellock6 » 1 person likes this post

The complete joke I did with Anton for /dev/null is: it's awesome for backups, just don't ask for a restore :)
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

So how about an idea for a repository that could be used to offload only copy job GFS restore points to. If a customer needs weekly, monthly, yearly, the cloud repo can get quite large. If we can add in a second repository, and have only those weekly, monthly, yearly GFS restore points go there, the cost wouldn't be as high with the lower tier of storage. I assume we could use file copy to move the GFS restore point wherever, but that doesn't allow it to be removed from the cloud repository.

Meeting some of the retention requirements today are easier to do onsite, but when we are talking about getting that data offsite, people balk at the amount of data created, the costs associated, and it's not always an automated process. Just looking for ideas on offloading some of those bigger restore points to cheaper storage. Thanks.
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS Retention Policy

Post by Gostev »

Yes, we have something like this in mind for the future versions. Thanks for confirming our thinking ;)
lennis40
Expert
Posts: 123
Liked: 3 times
Joined: Nov 11, 2014 11:03 pm
Full Name: Michael
Contact:

Re: GFS Retention Policy

Post by lennis40 »

Sounds great, thanks!
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

OK I have a question regarding GFS retention.

I have a backup job that is keeping 7 restore points, and when creating a backup copy job for GFS retention, I am forced to keep a minimum of 2 restore points. What is this setting for and can I not disable it or set it to 0? All I want is weekly, month and yearly as the main backup job is already keeping the daily points.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy »

This setting is for retention of the regular backup copy job chain (imposed by safety reasons).
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

Thanks foggy.

A few more questions for you on this topic:

1. Is there a way to have the backup copy job start automatically once the backup job is complete?
2. Should I have one backup copy job for all my backup jobs, or should I have one backup copy job per backup job?
3. Further to the above, does the backup copy job create one large file with whatever backup jobs have been selected?

Cheers
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy »

1. Schedule the sync interval to start a few minutes later than the original backup job starts. This way, backup copy job will monitor the newly created restore points and start transferring data immediately.
2. It's totally up to you (you can search forums for some considerations regarding this or that approach: dedupe, ease of management, etc.).
3. Yes.
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

OK great, thanks for answering those foggy.

When the backup copy job kicks off, which Veeam servers are responsible for processing the backup copy job data? Does the Veeam proxies do this, or is it the Veeam Backup Server?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS Retention Policy

Post by foggy »

Data is processed by Veeam data movers installed on the repository servers, proxies are not involved in backup copy jobs.
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

foggy wrote:Data is processed by Veeam data movers installed on the repository servers, proxies are not involved in backup copy jobs.
I had a look, and my Veeam B&R server is the one doing all the work (not the proxies as you mentioned). Is it possible to move the components that do this work to a separate server?
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

Also, how do I check on the backup chain to see where my weekly/montly/yearly backups are? At the moment I can only see the restore points specified in the backup job, and the restore points specified in the backup copy job.
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS Retention Policy

Post by Gostev »

btmaus wrote:I had a look, and my Veeam B&R server is the one doing all the work (not the proxies as you mentioned). Is it possible to move the components that do this work to a separate server?
Sure. Just create new backup repositories and proxies on another server (not a Veeam B&R server).
btmaus wrote:Also, how do I check on the backup chain to see where my weekly/montly/yearly backups are?
Go to Backups, right-click the job and open its Properties.
btmaus
Expert
Posts: 138
Liked: 10 times
Joined: Jul 17, 2015 9:02 am
Full Name: Glenn L
Contact:

Re: GFS Retention Policy

Post by btmaus »

I see one restore point has a "R" under retention. This Saturday was supposedly to be the first weekly (normal backup job keeps 7 restore points, backup copy keeps 2 restore points, 4 weekly, 12 monthly, 7 yearly), but it doesn't seem to have created it?
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS Retention Policy

Post by Gostev »

Feel free to open a support case to troubleshoot this.
Post Reply

Who is online

Users browsing this forum: DBerns, IvanK and 171 guests