Comprehensive data protection for all workloads
Post Reply
rawtaz
Enthusiast
Posts: 85
Liked: 15 times
Joined: Jan 27, 2012 4:42 pm
Contact:

Maximize number of restore points

Post by rawtaz » Mar 16, 2012 11:11 am

Hi,

Is there any way of telling Veeam B&R to keep adding restore points as long as there is free disk space (preferrably up/down to some percentage or amount of space left on the backup repository)? I have for example one backup repository that is purely dedicated to storing Veeam backups, and there's just no way I can accurately estimate how much data will be added each day, each month, or over a few years. For that reason I would want Veeam to just keep adding restore points as long as there's space left and then start overwriting the oldest ones.

Any way to do that? If not, what's the closest solution you can think of?

Thanks!

foggy
Veeam Software
Posts: 18245
Liked: 1558 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Maximize number of restore points

Post by foggy » Mar 16, 2012 12:14 pm

No such in-built functionality. Remember, that if you use forward incremental mode, you cannot simply delete the oldest restore point (VBK), as this will bring the whole following chain of increments unusable. This scenario looks a bit strange as it stands for the fact that the one using it has no strictly defined retention policy.
You can however, set a high number of restore points to keep and enable free space notifications in VBR. Once you get the notification, you can count the number of restore points and reconfigure your jobs with a bit lower ones.

Vitaliy S.
Product Manager
Posts: 22973
Liked: 1555 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Maximize number of restore points

Post by Vitaliy S. » Mar 16, 2012 1:15 pm

One more thing to consider - imagine that someone has accidentally filled this repository with data (ISO images or any other files), which led to a lack of free space for your backup job. So according to the logic above, Veeam backup job should adjust retention policy settings and wipe out existing restore points, right?

I don't believe this is a desired behavior, as it might led to a situation when all your backup restore points will be removed from the repository completely.

Hope this makes sense.

rawtaz
Enthusiast
Posts: 85
Liked: 15 times
Joined: Jan 27, 2012 4:42 pm
Contact:

Re: Maximize number of restore points

Post by rawtaz » Mar 16, 2012 1:19 pm

foggy wrote:No such in-built functionality. Remember, that if you use forward incremental mode, you cannot simply delete the oldest restore point (VBK), as this will bring the whole following chain of increments unusable. This scenario looks a bit strange as it stands for the fact that the one using it has no strictly defined retention policy.
You can however, set a high number of restore points to keep and enable free space notifications in VBR. Once you get the notification, you can count the number of restore points and reconfigure your jobs with a bit lower ones.
I should have mentioned that it is reverse incremental backups I'm doing :)

Strange, I dunno. The policy is simple; Keep as long of a history you can. We know that the amount of space available on the repository is large enough to hold a lot of/enough restore points even if there is a lot of changed data every day (within the reasonable bounds that are practical). So there IS space to hold it all, but I cannot predict whether it will be able to hold two months or five months. But whichever it is, since we have the space, lets use it. That's why I want to simply maximize the space usage.

By VBR do you mean Veeam B&R? I'll look for that free space setting. My testing so far indicates that it's possible to raise and lower the restore point setting without having to make a full backup, something I wish to avoid as one of my repositories is offsite. If so, your suggestion sounds good I think. Thanks!

rawtaz
Enthusiast
Posts: 85
Liked: 15 times
Joined: Jan 27, 2012 4:42 pm
Contact:

Re: Maximize number of restore points

Post by rawtaz » Mar 16, 2012 1:29 pm

Vitaliy S. wrote:One more thing to consider - imagine that someone has accidentally filled this repository with data (ISO images or any other files), which led to a lack of free space for your backup job. So according to the logic above, Veeam backup job should adjust retention policy settings and wipe out old restore points, right?

I don't believe this is a desired behavior, as it might led to a situation when all your backup restore points will be removed from the repository completely.

Hope this makes sense.
Yeah, you are right in this. But the backup repository has a lot more space than the entire datastore on the ESXi server, and users would have to *really really really* push a lot of changes on the VMs for it to fill up. There's no way that we'd be able to transfer anything but a small part of the changes in the backup window we have, due to the speed of the offsite connection.

But you are indeed right, one should have a minimum number of restore points, and if there isn't enough space to accomodate those, then the backup job should fail. In my case it's simply too unrealistic for it to be of any real concern (we also have other backup repos), but in general, yes.

Naturally the "accidental filling up the repo" could be done some other way as well, but.. Ok, so can we have these settings then? The "minimum restore points" and "fill it up, Scotty!" :mrgreen:

Vitaliy S.
Product Manager
Posts: 22973
Liked: 1555 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Maximize number of restore points

Post by Vitaliy S. » Mar 16, 2012 1:40 pm

"The minimum restore points" is already there! It is your retention policy configuration ;)

As to "fill it up, Scotty!", you can set retention policy to 999, that would allow you fill up your backup repository up to it's maximum. Also since you're using reversed incremental backup mode then once you receive a notification about lack of free space you can simply remove the oldest restore point (VRB) manually and continue running this job.

rawtaz
Enthusiast
Posts: 85
Liked: 15 times
Joined: Jan 27, 2012 4:42 pm
Contact:

Re: Maximize number of restore points

Post by rawtaz » Mar 16, 2012 1:53 pm

Vitaliy S. wrote:"The minimum restore points" is already there! It is your retention policy configuration ;)
Duh! :oops:
Vitaliy S. wrote:As to "fill it up, Scotty!", you can set retention policy to 999, that would allow you fill up your backup repository up to it's maximum. Also since you're using reversed incremental backup mode then once you receive a notification about lack of free space you can simply remove the oldest restore point (VRB) manually and continue running this job.
I see what you mean. I think that if doing this (999 restore points), one will at the time it fills up be able to determine a good restore points count based on how backups have been generated up until that point, set it to that and then be done with it for a while :)

Great, another far-from-common-use-cases corner-case question resolved for everyones future reference! 8)

tsightler
VP, Product Management
Posts: 5418
Liked: 2240 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Maximize number of restore points

Post by tsightler » Mar 16, 2012 1:59 pm

Sounds like a simple PowerShell script could take care of this. Just set the retention to 999, then run a PowerShell script once a day that checks free space and removes restore points based on thresholds.

Sord
Lurker
Posts: 2
Liked: never
Joined: May 28, 2012 1:24 pm
Full Name: Sord Data Systems

[MERGED] How to get Veeam 6 to overwrite old backups when ou

Post by Sord » May 28, 2012 1:27 pm

Hi, we're using Veeam B&R 6 and are trying to set it to reclaim disk space and to prevent it from failing due to no disk space,
Is there a way to tell it to overwrite oldest copy instead of trying to guess the number of VMs versions to keep?
Thanks

Vitaliy S.
Product Manager
Posts: 22973
Liked: 1555 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Maximize number of restore points

Post by Vitaliy S. » May 28, 2012 1:56 pm

There is no way to manually force Veeam B&R overwrite oldest files in the backup chain. Please note that the number of restore points to keep is controlled by configured retention policy settings, so in order to reclaim some disk space, you may want to adjust these settings and keep less restore points for the VMs in the backup job.

dellock6
Veeam Software
Posts: 5732
Liked: 1622 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Maximize number of restore points

Post by dellock6 » May 28, 2012 4:53 pm

Furthermore, I think there is a problem with deduplication and compression effects: Veeam cannot know beforehand how big a new backup will be before applying these optimizations, so is not possible to know how many restore points need to be deleted to fit the new one into the storage.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Bowie1979
Novice
Posts: 6
Liked: never
Joined: Oct 18, 2012 1:00 pm
Full Name: Scott Fuller
Contact:

Re: Maximize number of restore points

Post by Bowie1979 » Oct 18, 2012 2:46 pm

I'd just like to hop onto this old thread if I may :)

Is veeam able to cope with (potentially) hundreds of reverse incremental restore points like this? Rather than fiddling about with Weekly and Monthly full backups, I am thinking of filling our backup drive with as many reverse incrementals as I can. I understand it would take longer to process a restore from an old reverse incremental, but wanted to check whether or not it's a good idea.

thanks,
Scott

dellock6
Veeam Software
Posts: 5732
Liked: 1622 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Maximize number of restore points

Post by dellock6 » Oct 18, 2012 3:03 pm

Well, statistically, the older the restore point less chance you would use it to restore something from.
I've seen once a backup job of a file server with one full year of restore point (yes, 365 restore points) used only for regulatory purposes. Never used any restore point older than a month...

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Bowie1979
Novice
Posts: 6
Liked: never
Joined: Oct 18, 2012 1:00 pm
Full Name: Scott Fuller
Contact:

Re: Maximize number of restore points

Post by Bowie1979 » Oct 18, 2012 3:13 pm

That's what I was thinking. I really can't see myself using restore points beyond a month either, buts its nice to be able to say we have the older backups.

rawtaz
Enthusiast
Posts: 85
Liked: 15 times
Joined: Jan 27, 2012 4:42 pm
Contact:

Re: Maximize number of restore points

Post by rawtaz » Sep 01, 2013 3:31 pm

Vitaliy S. wrote:As to "fill it up, Scotty!", you can set retention policy to 999, that would allow you fill up your backup repository up to it's maximum. Also since you're using reversed incremental backup mode then once you receive a notification about lack of free space you can simply remove the oldest restore point (VRB) manually and continue running this job.
Hi again,

Following up: I have a reverse incremental job in B&R 6.5, that I'm running 13 times a day. For e.g. 90 days that gives 13*90 = 1170 restore points. I tried setting the job's "Restore points to keep on disk" to 1500, but it reverted to 999.

Reading the User Guide, it sure seems that when the specified number of restore points are hit, the next backup run will delete the oldest restore point (quote: "If this number is exceeded, the earliest restore point will be deleted").
But that doesn't make sense with what you said earlier, the "you can set retention policy to 999, that would allow you fill up your backup repository up to it's maximum" - the latter seems to suggest that a value of 999 will not delete restore points. Can you clarify?

Thanks!

tsightler
VP, Product Management
Posts: 5418
Liked: 2240 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Maximize number of restore points

Post by tsightler » Sep 01, 2013 4:59 pm

999 restore points is the maximum, older than that and they will be deleted. I think you're unlikely to get to 999 restore points without serious performance issues. I did extensive testing on 6.5 and after about 750 restore points or so performance really fell off, taking 10+ minutes per VM and this was on just a handful of small VMs with limited change. I haven't retested with 7.0 however.

You might be better off using copy jobs since they can easily provide 90 days of daily backups while allowing you to continue running 13 backups a day for your main job. Keep those for 30 days, and then keep dailies for 90 or 180 days in a Backup Copy.

Post Reply

Who is online

Users browsing this forum: DGrinev, FSenturk, nielsengelen, staley and 62 guests