Backup files not deleted according to retention policy

[bshubinsky]

My Veeam is currently set to 7 rollback points and for it to do a rollup on Sundays for the incremental. However, it does not appear to be rolling up (so I've got a ton of individual backups) and on top of that I've got 20 restore points instead of the 7.

Has anyone run into this problem?

[Gostev]

Keep in mind that incremental backup mode by definition requires all previous backup files (full and incremental) to be available to make restore from incremental backup possible. So, as long as last incremental backup file in backup chain must be restorable according to set retention policy, whole respective full + incremental backup chain will have to stay on disk (resulting in more backup files on disk then you may have expected). After last incremental backup file in backup chain falls out of your data retention policy (which happens only when retention policy reaches next full backup file), whole previous full backup chain is removed at once.

On the other hand, reversed incremental backup mode (or incremental mode with transform enabled, once transformation happened) removes older restore points immediately, as soon as they fall out of retention policy. This, and the fact that these backup modes allow to keep only one backup file on disk no matter of how long your retention policy is, make them a few times more space efficient than traditional incremental backup.

[rowdy]

I'm also seeing this in version 5.0.1:

I have the setting 'keep number of rollback points' set to 14, backups run every weekday (+Saturday, see below).
I'm making forward incremental backups with synthetic full every Saturday.
Furthermore, I have a full (normal) backup setup every first Saturday of the month.
So, I would expect to have a VBK file, then 5 VIB files (mo-fr) and then another VBK file with 5 more VIBs.
This is working fine, however, I have an 'infinite' number of restore points now and it just keeps creating VBK with VIB chains and never cleans up anything.
I manually delete the VBK/VIB chains over a month old but that's not the best way.

Do I have something set-up wrong or might there be a bug?

[Gostev]

So, I would expect to have a VBK file, then 5 VIB files (mo-fr) and then another VBK file with 5 more VIBs.

No, this is exactly what most people tend to think for some reason, but it is incorrect assumption in case of traditional incremental backup... please read my explanation above carefully.

I guess I should try to visualize this based on your specific example...

Depending on specific day, in "best" case, with your retention and schedule you will have 14 files on disk (since you said your retention is 14 rollbacks):

Code: Select all

F I I I I I F I I I I I F I 

However, in worst case, when small "i" is your 14th rollbackup point, you will have 20 files on disk:

Code: Select all

F I I I I i F I I I I I F I I I I I F I

The whole full backup chain with small "i" in the end will only be removed by retention after latest incremental is created sucessfully, and then we are back to having 14 files on disk.

[Gostev]

That said, with your 14 point retention policy and 6/7 backup schedule, you should not have more than 4 fulls on disk at any given moment. If you have more than 4 fulls, I would recommend opening a support case and providing full logs. It should be clear from logs if data retention policy is being processed correctly (or why it is not processed). One thing that is unusual in your case is that you only do backup 6 days of the week, and there is always a chance of bug with non-default settings.

[rowdy]

Thanks for your reply.

I understand your drawing, however I don't fully understand it then. I will leave everything running a bit longer and stop manually deleting files and see what Veaam does

[colintyrrell]

[merged]

Hi,
h0pefully a simple question.

When backing up my servers I have selected 14 restore points.
But when I do a restore, it seems there are jobs with considerbaly more than 14 points..24 points, 23 points etc.

What am I doing incorrectly?

[Alexey D.]

Colin, are you using incremental backups?

[colintyrrell]

I am Alexey

[colintyrrell]

Hi,
should i create a new post on this?
It really is becoming an increasingly critical issue.
At the moment I seem to have little choice but to create new jobs in order to free up space.

For instance the new job is set yo use 7 restore points. after 9 days I'm up to 9 points.
Am i doing something silly?

[Alexey D.]

Hello Colin,

Please read this thread, it explains your case: v5 Restore Point Bug?

[Vitaliy S.]

Here is another topic explaning this behaviour, please have a look: Job Set for 14 Mount Points - 46 exist

If you have little space on your target storage, then you should probably switch to Reversed Inceremental backup mode.

Please have a look at the Excel table showing all the Pros and Cons of different backup modes: [FAQ] v5 : Frequently Asked Questions > Answers

[bshubinsky]

[merged]

Quick question: if I'm doing incremental backups with Veeam, the restore points to keep on disk number; does this refer to full backups, or incremental days? The reason is this:

I'm currently running incrementals with 4 days. I'm starting to run out of space - I tried doing a full backup to have it roll up the previous incrementals but it's still keeping them out there, which isn't helping my situation.

[Chriss]

[merged]

We cuurently backup all our virtuals to disk using Veeam. These backups, are then copied using MS DFS to our DR site.
Every weekend, we write the disk backups to tape, and move them off site. My problem is, I only want 8 backups on site here at Corporate, as we have limited disk space on our EVA. When I look at the backups, some of the servers have 32 and more restore points. What would be the best settings to ensure this.

[chrmol]

[merged]

Hi.
We are running Incremental jobs with weekly active full ones pr day and with retention set to 21 - but files older than 21 (days) doesnt seam to be deleted in the filesystem.
Have we misunderstanded the meaning of retention (I know that it isnt days! - but number of backups

[chrmol]

This is really starting to be annoying - our backup destination ran full this night. We have retention set to 21 - but a couple of jobs now have 6 Full backup and all the incremental in between!
It works on some jobs, and not on others.
I have now done a manually clean-up (deleted the older files) and update to latest version 5.0.2 – ill keep an eye on the files the next couple of days.

[Vitaliy S.]

Christian, please look through this topic for better understanding retention policy and Forward Incremental backup mode. After that please make changes to your retention pocily, so you do not fill up the destination datastore.

[chrmol]

I already did - if i have misunderstanded the way retention works please give me some advice.

[Vitaliy S.]

Chrisitian,

Basically with Reversed Incremental mode the retention policy is a straight forward thing. If you define 7 restore points to keep, then you'll have 1 VBK and 6 VRB files.

However, with Forward Incremental backup mode it is getting a little bit more complex. Generally speaking you'll have twice more restore points available on the disk compared to the retention policy settings.

Tom has posted a pretty detailed explanation on the previous page of this very topic. In addition to this, please look through these topics more carefully: v5 Restore Point Bug? and Job Set for 14 Mount Points - 46 exist

On top of that, what is your retention policy now (still 21?) and how have you scheduled your backup jobs?

Thanks.

[chrmol]

I believe that the reason is that I have some deleted VMs inside thoose jobs that won't cleanup!
I have updated to latest veeam version which i can se that is able to handle this (deleted VM retension days) - I hope that it is solved for now.
I'm able to conclude this in about a week or so.

[pizzim13]

[merged]

I am having the same issue. I have a backup job with a retention policy of 14 but Veeam shows 26 restore points. Running 5.0.2.230

[MattR]

[merged]

We have Veeam Backup & Replication version 5.0.1.198 (32bit) installed on Windows Server 2008 that is backing up from ESXi 4.1 to a Linux back-end. We have it configured to do Synthetic incremental backups for VMs with a limit on the restore points to keep on disk set at 28. However, we have found that the backups are not being purged as they should be.

Information about the server properties:
Elevate account to root is selected, but 'add account to sudoers file automatically' is not (attempts to add automatically always fail, the account was added manually and is a member of wheel which we have set for not requiring password for now).

Backup files are being created as Root:Root with a mask of 644. The Linux server build is a Gentoo 2.6 Kernel.

Please provide assistance on what else in the configuration there may be to have the old files properly purged.

Thank you,


--

[M70374]

[merged]


Hello,
we have two VMs running VBR 5.0.2 in site locations where small ESXI 4.1.0 clusters are placed. The setting "restore points to keep on disk" = 7 is being ignored. I.e. one VM backed up has a vbk file of 2011-04-30 and 33 vib files. I modified the setting to 14 days and re-run the job without errors but the amount of restore points was not corrected.
I am really confused since we use VBR in the same version for our big central clusters in our main location without these problems. The only difference is that it is a VM using option Virtual Appliance in the site locations while we have physical servers in our main location using option "Direct SAN Access".

[Alexey D.]

Tobias, most probably you didn't specify the schedule to create synthetic full backup. Since you have only 1 VBK file, older incremental points can not be removed according to retention policy, because they cannot exist without "parental" VBK. Please go through the current topic if more information needed.

[J1mbo]

[merged]

My backup jobs don't seem to be ever purging any old copies - is there some trick to this? My jobs specify the default 14 restore points, yet the client is showing 20+ on some VMs (basically they've not been purged since job creation, it seems).

Many thanks

[kratos31]

[merged]

Hi,

Our backup job is set to keep 14 restore points, at the moment we have 20.
Its causing a bit of a problem it terms of disk space.

Should the last backup not be removed once we get to 14?
thanks

[habibalby]

[merged]

Hi,
I have setup backup jobs with settings:

Restore Points to Keep on Disk: 5 Days
Delete VMs Retention Period: 7 Days

I have noticed that when I run the Backup after 7 Days or even 10 / 15 Days, still the old backup exists and are not being deleted. This causes the Targets become Full and I cannot run Backup anymore..

How can I get rid of the old Backup if the backup expired based on the Retention Period?

Thanks,

[JorisK]

[merged]

Hi,

We have a job that should be having max 5 restore points, and a max retention of 7 days. Still, the job has 10 restore points availible for restore. I have a lot of jobs like that. Anyone any idea why this is possible?

I have to mention that we use sequenced jobs and some of the jobs got started manually using the "Start Job" button in the past days.

Regards,
Joris

[pmolenbrugge]

hi all,
had the same situation. Our problem was that the day we chose to compact the incrementals (saterday) was NOT in the schedule for the job (only mon-fri).One we added satuday problem was solved. Kind of strange however that veeam lets you create the job this way. Would be nice to have a check implemented that avoids this kind of errors.
pim

[Gostev]

Yes, I believe this is fixed in v6. Thanks!