Discussions specific to the VMware vSphere hypervisor
Post Reply
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

How many restore points does Backup Copy keep?

Post by readie » Oct 08, 2014 12:04 pm

In the TARGET window for a backup copy job, there is a field for 'number of restore points to keep', and then lower down 'Keep the following restore points for Archive Purposes'.
I am running this only once a week, so ask to keep 5 restore points - I assume for the past 5 weeks.
I ALSO want to keep 2 monthly backups, 2 quarterly backups, and 10 yearly backups (if we ever get that far!).
However, when I now want to restore from backup copy, I can see the past 5 weekly backups, and one on 6/08/2014 and one on 31/07/2014 - only about a week apart.
I can't see any that might be monthly, or quarterly . . . and I think this has been running since April (though cannot be certain).
Have I confused it by asking for only 2 monthly, and only 2 quarterly?
Any help please. Thanks,
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 08, 2014 1:09 pm

Bob, what are the days the monthly and quarterly backups are scheduled to be created at and what day the oldest restore point in the chain is currently dated? You can review this thread to get better idea of how GFS restore points are created, however with complete backup copy job settings I will probably be able to explain what you currently have on disk.

readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: How many restore points does Backup Copy keep?

Post by readie » Oct 08, 2014 1:27 pm

Thanks Foggy. I have just left the default as 'first Sunday in the . . . month, quarter, year' (though I have registered a feature request to enable academic establishments to choose a date for annual archives between academic years - in July for us).
Dates of actual restore points are
Thurs 31/07
Wed 6/08
Thurs 4/09
Wed 10/09
Wed 17/09
Wed 24/09
Thurs 2/10
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 08, 2014 2:14 pm

GFS restore points are created from the oldest restore point in the chain, when it reaches the day specified in the settings (i.e. first Sunday...). On that day, Veeam B&R takes the current full backup and marks it as a weekly/monthly/etc. However, if there's no full backup on that day (like in your case, since you've probably started the job on Wednesday initially and there is no full backup dated on Sunday), it takes the nearest full (created next Wednesday, in your case).

So, since the first Sunday of the August is 3/08, you have the monthly restore point dated Wed 6/08. The next monthly will be created based on full dated 10/09. Btw, restore points are marked as monthly ("M") or quarterly ("Q") right next to the restore point description in the backup properties.

I'm not sure regarding the "Thurs 31/07" restore point though. How it is marked in the restore points list in the backup properties window?

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 08, 2014 2:57 pm

Just a thought: have you enabled GFS retention initially when creating the job or at some point later?

readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: How many restore points does Backup Copy keep?

Post by readie » Oct 08, 2014 3:07 pm

Initially, as I use this just as my 'archive' backup - it is on the same physical system as our regular backups, rather than being a copy to another site.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 08, 2014 3:27 pm

Got it. So how that restore point is marked in the backup properties?

readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: How many restore points does Backup Copy keep?

Post by readie » Oct 09, 2014 8:36 am

(Sorry - did quite a long reply yesterday, but it hasn't appeared??)
I hadn't seen the properties box you described, but now I see the M,Q etc it makes more sense.
31/7 MQ
7/8 M (only one week later, but I understand that it is immediately after the first Sunday)
4/9 R
then 11/9, 18/9, 25/9, 2/10, 9/10 incrementals with no letter in the retention column.
There is no M for September, even though there have now been a couple of weeks in Oct.
Actually, I understand. I asked it to keep just two monthly, and it is keeping the oldest two, whereas I had hoped it would be the most recent two (as it does with normal backup restores - if I say keep 5, it keeps the most recent 5).
In order to conserve repository space, what I was hoping for was . . . say near the end of December one year . .
Annual from start of Jan (then 6 month gap), two most recent quarterlies start of July and Oct, two monthly from start of Nov and Dec, and weeklies through December. That gives gradually increasing gaps going backwards . . . 1W, 1W, 1W, 1M, 1M, 3M, 6m, 1Y, 1Y, 1Y etc. I think?
But it takes such a long time before I find out whether it is actually doing what I want - and I am concerned it is not keeping the most recent two monthlies?
Thanks for your continued help.
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 09, 2014 9:40 am

readie wrote:There is no M for September, even though there have now been a couple of weeks in Oct.
As I said, September's monthly will be created when the full backup reaches 10/09 (currently it is 4/9, so after the next job run). The monthly mark will be cleaned from the "31/7 MQ" full, it will be retained as quarterly full only.
readie wrote:But it takes such a long time before I find out whether it is actually doing what I want - and I am concerned it is not keeping the most recent two monthlies?
The way it works, it offloads GFS points after reaching the specified regular retention only. So a monthly backup is created for the previous month (after 5 weeks elapse). You can lower regular retention to have a shorter chain, this will result in more recent GFS points.

readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: How many restore points does Backup Copy keep?

Post by readie » Oct 17, 2014 8:48 am

Hi,
It is now creating its next full backup (M?) but I have just realised that we are running out of HD space on the Repository . . .
Since the Archive copies are just that, is there any way I can copy (say) my Q, and oldest M backup onto another HD (or even another Repository) without confusing VEEAM backup?
Just checked, and it looks like it WON'T have enough space to finish this transform. I can't 'pause job' while I try to create space can I? How 'gracefully' is it likely to crash out?
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 17, 2014 8:52 am

Yes, you can safely offload older GFS restore points to another location. Just keep in mind that corresponding records will still be displayed in Veeam B&R console until wiped out from the database according to retention.

pkelly_sts
Expert
Posts: 577
Liked: 64 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: How many restore points does Backup Copy keep?

Post by pkelly_sts » Oct 20, 2014 3:21 pm

foggy wrote:...

Btw, restore points are marked as monthly ("M") or quarterly ("Q") right next to the restore point description in the backup properties.

I'm not sure regarding the "Thurs 31/07" restore point though. How it is marked in the restore points list in the backup properties window?
Sorry for jumping in on this thread, but where exactly can I find the properties referenced above? I can't figure out which "Backup properties window" is being referenced here?

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 20, 2014 3:35 pm

Paul, click the Backups node, right-click the corresponding backup in the right pane and select Properties.

pkelly_sts
Expert
Posts: 577
Liked: 64 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: How many restore points does Backup Copy keep?

Post by pkelly_sts » Oct 21, 2014 8:10 am

Ahhhh, now I see - in around 2 years of using Veeam I've /never/ seen that view! :) I've always just clicked on the Disk or Imported sub-nodes.

What would be REALLY useful is if a delete option could be added here. Sometimes we have failed synthetics (generally our own fault rather than Veeams) which causes older backups to be kept longer than originally intended due to longer resulting dependency chains and it would feel so much better deleting them in such a way as Veeam knowing about it rather than pulling it out from under it by deleting manually...

Paul

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 21, 2014 9:20 am

pkelly_sts wrote:What would be REALLY useful is if a delete option could be added here. Sometimes we have failed synthetics (generally our own fault rather than Veeams) which causes older backups to be kept longer than originally intended due to longer resulting dependency chains and it would feel so much better deleting them in such a way as Veeam knowing about it rather than pulling it out from under it by deleting manually...
Paul, could you please clarify the use case? Remember, you cannot delete restore points from the middle of the forward incremental chain, this will result in inconsistency of the entire chain and inability to restore from the left over restore points. Once the next full will be created, excessive restore points will be automatically deleted, according to your retention settings.

pkelly_sts
Expert
Posts: 577
Liked: 64 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: How many restore points does Backup Copy keep?

Post by pkelly_sts » Oct 21, 2014 10:45 am

My use case has two variations, one of which I burned myself with last week which caused me some (self-inflicted) grief.

Case 1:
Normal local "Forever-incremental" backup job, each full around 1.7Tb being stored on a 5.5Tb disk partition.
7 Retention points, Synth full on a Friday. Incrementals up to around 25Gb

This leaves capacity for the 7 RPs plus the extra older week until the last incremental within the retention is no longer dependent on the oldest full.
Most importantly, backups are sent to tape each night (so incrementals only sent to a single tape swapped each day during the week then the synth full to multiple tapes over the weekend).

All works fine until there's a problem which causes a synth full to fail (generally not Veeams fault, most recent case was a network issue).
This means there are now more incrementals dependent on the oldest full which effectively causes it to be kept around for another week meaning we now require capacity to hold *4* fulls which we obviously don't have so I then manually delete the oldest full (already have it on tape) and the incrementals that are dependent on it (so useless without it anyway).

Case 2:

Very similar to case 1 but this time revolving around Backup Copy jobs to a remote site (in this example of the same data/job)
I my case I ran out of space, can't remember exactly why, but from what I've seen, even if you lower the retention figure the older files are only deleted /after/ the current job completes, but if the current job can't complete due to lack of space then you end up in a vicious circle.

In my case I deleted the oldest 2 fulls and what I thought were dependent incrementals but the copy job then started complaining about incrementals older than the retentions I'd configured being missing & it all got into a bit of a mess ultimately needing me to move all the remaining files to one side (just in case) and start the whole 1.7tb copy job again over the 100mb link and let it run over the weekend. In our situation a re-seed by creating a new backup copy, dumping to tape, moving the tape & restoring at the other end would have taken as long (if not longer) anyway so I just let it get on with it.

Seeing info coming out about the new style backup of v8 and re-viewing explanations of how the forward/reverse incrementals work I think I didn't (and possibly still don't) fully understand how the backup files tie together on a backup copy job so I need to find some time to get my head around that properly again...

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 21, 2014 11:37 am

Thanks for clarification, I got it.
pkelly_sts wrote:Seeing info coming out about the new style backup of v8 and re-viewing explanations of how the forward/reverse incrementals work I think I didn't (and possibly still don't) fully understand how the backup files tie together on a backup copy job so I need to find some time to get my head around that properly again...
Basically, backup copy job chain is just an incremental chain containing VBK file as the first and the oldest restore point in the chain and a set of subsequent increments (VIB files). If you have GFS retention enabled, each GFS restore point is stored as a separate VBK file and does not count into the basic retention of the job, so can be safely deleted manually without consequences for the job operation (can be changed in v8, though).

pkelly_sts
Expert
Posts: 577
Liked: 64 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: How many restore points does Backup Copy keep?

Post by pkelly_sts » Oct 21, 2014 1:38 pm

Thanks, makes perfect sense & I'm not sure exactly how I managed to so badly break this particular copy job but I'll do some testing on a single small VM I think & have a play to ensure I fully understand it...

foggy
Veeam Software
Posts: 18552
Liked: 1606 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How many restore points does Backup Copy keep?

Post by foggy » Oct 21, 2014 1:40 pm

Feel free to ask, if after your testing anything still requires clarification.

Post Reply

Who is online

Users browsing this forum: RobMa, veremin and 26 guests