v5 Restore Point Bug?

[ctchang]

I've setup the backup schedule to retain 7 days of restore points, but I found Veeam Backup won't delete old ones, it's just continously growing.

See attached.





Note there are more than 7 vib.




Note I manually deleted the one from Nov7 today (it's 120GB), it's a full backup copy vbk. Anton, can I do that? (I mean manually remove it?)
Will I lose any data on Nov 7? I thought there was going to be a vib for Nov 7 and also a Synthetic Full (vbk) for Nov 7, so total 2 files. Somehow the weekly Synthetic Full on Sunday (Nov 7) contains the incremental vib as well? Now, since I've deleted the weekly Synthetic Full backup (ie, Nov7), will I encounter problem for tomorrow morning backup? (6am), as there is a missing file in the backpu chain (ie, Nov 7).

Thanks.

[tsightler]

You don't have any full backups in the last 7 days so what backup do you think Veeam should delete? If it deleted your full backup then the incrementals wouldn't do any good since they depend on the full, and if it deleted any of your older incrementals, then the later incrementals wouldn't be any good since later incrementals are dependent on previous incrementals. If you really want a 7 day retention when using incremental backups then you'll need a full backup at least every 7 days, either a true full or a synthetic full. Or you could perform a transformation to reverse incremental after each backup or just use reverse incremental backup mode.

[tsightler]

Note I manually deleted the one from Nov7 today (it's 120GB), it's a full backup copy vbk. Anton, can I do that? (I mean manually remove it?)
Will I lose any data on Nov 7? I thought there was going to be a vib for Nov 7 and also a Synthetic Full (vbk) for Nov 7, so total 2 files. Somehow the weekly Synthetic Full on Sunday (Nov 7) contains the incremental vib as well? Now, since I've deleted the weekly Synthetic Full backup (ie, Nov7), will I encounter problem for tomorrow morning backup? (6am), as there is a missing file in the backpu chain (ie, Nov 7).

I didn't see this part of your question first. Yes, of course you will loose the data from Nov 7th, and actually, from all of your incrementals as well since they were incremental from the full backup, that's pretty much the definition of incremental.

I think you need to think through the process a little bit. You are using incremental backups but asking for 7 days of retention, but only running fulls once a week. This is pretty much not really possible, incrementals require a full backup to be any good, so you have the following:

Sun -- Full
Mon -- Incr
Tues -- Incr
Wed -- Incr
Thur -- Incr
Fri -- Incr
Sat -- Incr
Sun -- Full

OK, so you have a 7 day retention and now it's the second full backup, which, according to your logic means that Veeam should delete the backup from the previous Sunday. The problem with this is that, if it deletes the first full backup from Sunday, then all of the incremental backups from Mon-Sat will not be any good which means you wouldn't really have 7 days worth of retention.

So, Veeam does the correct thing, it waits until the 14 day mark, which is really the first time that it can delete any files and still keep you with the full 7 days retention. If it deleted files any sooner, you would have less than the 7 days of restore points. I guess you could say, when using incrementals backups, where restore points are dependent on previous restore points, that setting the retention is really more like setting the "minimal" retention. Veeam will manage your backups to make sure you retention never falls below the number of days you requested, but you might sometimes have to have more days. This is simply the nature of incrementals, and has been since the beginning of time, the granularity of your retention policy is limited by how often you run full backups.

If you want to keep exactly 7 days of retention then you need to use reverse incrementals, or perhaps regular incrementals with a full/synthetic full twice a week, which would get you really close.

[ctchang]

Many thanks again Tom.

So are you saying if I set retention to 7 days, then Veeam will only start to delete on the 15th ? See following.

Sun -- Full--01
Mon -- Incr
Tues -- Incr
Wed -- Incr
Thur -- Incr
Fri -- Incr
Sat -- Incr-07

Sun -- Full-08
Mon -- Incr
Tues -- Incr
Wed -- Incr
Thur -- Incr
Fri -- Incr
Sat -- Incr-14

Sun -- Full--15

Will it start to delete (01-07) only after making the 3rd full backup on Sun (Sun -- Full--15)?

[Vitaliy S.]

Hi Jack,

Yes, that's correct. Please have a look at User Guide (page 13) for more description about retention policy, though Tom's explanation was really good!

[tsightler]

Sun -- Full--01
Mon -- Incr
Tues -- Incr
Wed -- Incr
Thur -- Incr
Fri -- Incr
Sat -- Incr-07

Sun -- Full-08
Mon -- Incr
Tues -- Incr
Wed -- Incr
Thur -- Incr
Fri -- Incr
Sat -- Incr-14

Sun -- Full--15

Will it start to delete (01-07) only after making the 3rd full backup on Sun (Sun -- Full--15)?

It would certainly delete it after the third Sunday full backup, it might possibly delete it after the second Saturday incremental backup since technically keep Sunday - Saturday is enough meet 7 days retention, I'm not sure of the specifics of the algorithm in this case.

[ctchang]

Thank you very much again.

[Gostev]

Please also note that using previous full backup chain transform option on the backup mode selection dialog allows to convert previous full backup chains into rollbacks going back from current full backup. This in turn allows to immediately delete older rollbacks according to your retention policy. There is no more need to wait until last increment of previous full backup chain is needed by retention, and then delete the whole chain. And, of course, more importantly is that this allows you to keep only one full backup on disk no matter how long your retention policy is (which translates into dramatic disk space reduction for your entire backup set).

[MSoft]

Hi, I also have a question regarding backup file retention. We are running Veeam v5.0.1. The first backup was active full; the job was then changed to synthetic full. We now have 2 .vbk files – the original active full & the later synthetic full. As the job is configured to retain 1 restore point, why was the first .vbk file not deleted on completion of the synthetic full? Is it OK to manually delete the first .vbk file, and if I do, will the number of ‘Restore points’ under the Veeam Backups tab automatically reduce to one?

[Vitaliy S.]

It is expected behaviour as retention policy is applied before the transform operation takes place. You can manually remove that file but the record will still remain in the SQL configuration database as an available restore point.

[MSoft]

Thanks Vitaliy, the first .vbk file was deleted during last night's incremental backup. Would it not make sense to clean up expired files at completion of the backup?

[Vitaliy S.]

Yes, it might be worth applying retention policy after the transform operation, we'll see... Thanks!

[damocue]

[merged]

Hi,

Maybe this is an old question, but I have a confussion with the restore points. I understand the number of restore points as the number of backups they are saved on disk. For example, I backup 5 days pro week with 4 incremental backups and 1 synthethic full backup, and I set 10 restore points. This would give me 2 weeks of backup, no?

So, before a full backup I should have this on disk, in a timeline (FB: Full backup | IB: Incremental backup):
FB - IB - IB - IB - IB - FB - IB - IB - IB - IB

And after a full backup:
IB - IB - IB - IB - FB - IB - IB - IB - IB - FB

And 2 days after a full backup:
IB - IB - FB - IB - IB - IB - IB - FB - IB - IB

No? But I have this on disk, 2 days after a full backup:
FB - IB - IB - IB - IB - FB - IB - IB - IB - IB - FB - IB - IB

Ah, and on the User Guide, is correct this paragraph? I copy-paste it:

Specify the number of restore points that should be kept on the disk. If this number is exceeded, the earliest restore point will be deleted.

I think it should be "the oldest", not "the earliest" restore point to be deleted.

Regards,
David

[Gostev]

Hi,

Incremental backups cannot exist without full backup they are going from. This is fundamental thing about incremental backup that has nothing to deal with Veeam specifically. Until you need to be able to restore from latest incremental in chain according to your retention policy, whole chain (full backup and all incrementals) must be kept. Then, whole chain is removed at once.

The beauty of reversed incremental backup is that it has no such limitation, and older restore points can be removed immediately as needed.

Yes, looks like a typo.

Thanks.

[damocue]

Ok, thank you very much!!

[danieln]

[merged]

Hi,

I am running veeam B&R 5.0.2 against ESX4.1. Veeam server is a guest on vSphere/vCenter backing up over network.
At veeam full backup not all of the prior VIBs get erased. The behaviour is present after I switched the target from CIFS to linux target + agents.

I don't want after each full backup to go into the linux box and erase obsolete VIBs.

Any starting point on what may happen here?

Thank you,
Daniel.

[danieln]

HI again,

I am sorry, my problem is not related with this trail of non issue/missunderstanding on how Veeam incremental backup works.

I clearly have VIBs that remain un erased, not all of them, after a full backup.
I have : 1 restore point to keep, I know incrementals cannot be erased as they depend on fulls.
I run full on weekend, 1 restore point to keep. After a full backup, instead of having 1 VBK file, I still have some of the old VIBs still present and VIB files date time stamp updated by the backup process.

The behaviour is different (I mean it works correctly) when I back up on CIFS share.

Example of a DIR by date for one of the backups:

Code: Select all

07/09/2011  07:18 AM     1,109,991,936 Storage2011-07-08T090056.vib
07/09/2011  07:18 AM     2,118,903,296 Storage2011-07-07T130044.vib
07/09/2011  07:18 AM       454,446,592 Storage2011-07-06T170107.vib
07/09/2011  07:19 AM       793,091,584 Storage2011-07-06T130051.vib
07/09/2011  07:19 AM       405,336,064 Storage2011-07-05T130037.vib
07/09/2011  07:19 AM       492,158,976 Storage2011-07-05T090050.vib
07/09/2011  07:19 AM       455,201,280 Storage2011-07-04T090040.vib
07/16/2011  06:30 AM     1,047,784,448 Storage2011-07-15T090049.vib
07/16/2011  06:30 AM     1,970,232,320 Storage2011-07-14T090038.vib
07/16/2011  06:30 AM     1,284,527,104 Storage2011-07-13T170107.vib
07/16/2011  06:31 AM     1,180,572,672 Storage2011-07-12T170052.vib
07/16/2011  06:31 AM       995,255,808 Storage2011-07-11T130040.vib
07/23/2011  06:20 AM     1,412,174,336 Storage2011-07-21T090034.vib
07/23/2011  06:20 AM       597,952,000 Storage2011-07-20T170101.vib
07/23/2011  06:20 AM       731,183,104 Storage2011-07-20T130048.vib
07/23/2011  06:21 AM     1,405,416,448 Storage2011-07-19T090053.vib
07/23/2011  06:21 AM       557,806,592 Storage2011-07-18T170056.vib
07/30/2011  06:20 AM     1,553,555,968 Storage2011-07-27T170058.vib
07/30/2011  06:20 AM     1,388,224,000 Storage2011-07-27T090058.vib
07/30/2011  06:21 AM       963,284,480 Storage2011-07-25T130051.vib
07/30/2011  06:21 AM     1,063,502,848 Storage2011-07-25T090046.vib
                      08/06/2011  06:29 AM   381,070,536,704 Storage2011-08-06T000029.vbk
08/08/2011  09:12 AM     3,043,616,256 Storage2011-08-08T090102.vib
08/08/2011  01:06 PM     1,029,089,792 Storage2011-08-08T130037.vib
08/08/2011  05:09 PM       960,551,424 Storage2011-08-08T170126.vib
08/09/2011  09:09 AM     1,414,206,976 Storage2011-08-09T090045.vib
08/09/2011  01:07 PM       966,099,456 Storage2011-08-09T130036.vib
08/09/2011  05:09 PM     1,170,447,872 Storage2011-08-09T170104.vib
08/10/2011  09:10 AM     1,330,644,480 Storage2011-08-10T090050.vib

Veeam says 8 restore points but as anyone can see, there are far more than 8 files.

Anyone experiencing the same issue?

Thank you,
Daniel.

[Vitaliy S.]

You definitely shouldn't have orphaned VIB files - the ones that are older than the latest VBK file (08.06.2011). Judging by Veeam backup console, those files (old VIBs) should have already been erased that is why they are not visible as restore points... Please contact our support team for further investigation.