Feature request: Please bring back the 'ForceDeleteBackupFiles' feature in V12!

[martinic]

Feature Request: enable B&R V12 to remove the oldest restore point from backup disk when there's not enough space to add a new restore point.

When working with rotated media (for example one backup disk for each week day), for B&R V11 there is a registry key that enables the feature to automatically remove oldest restore point(s) from backup disk when it needs more space.

Code: Select all

[HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication]
"ForceDeleteBackupFiles"=dword:00000003 

Having had communication with Customer Support (Case 05965686) they confirmed that this feature has been removed from V12.

There are GUI options now, but you can't configure V12 to work like V11 with its reg key feature:
a) continue backup chain -- keeps adding files to backup disk until no space is left and backup fails
b) remove all backups for this job -- this deletes all restore points from this backup job
c) remove all backups for all jobs -- this removes all backups from all backup jobs

This means with V12 you are forced to:
a) manually clear old restore points each time a backup disk is full, which is unworkable
or
b) and c) only have ONE restore point per backup disk; which means for a week day rotation your oldest backup is just from 5 days ago

Please bring back the 'ForceDeleteBackupFiles' feature in V12!

[Gostev]

@Egor Yakovlev please look into this, I'm not sure I understand what exactly was lost in V12 considering we (presumably) just moved all the existing registry options into the UI... or at least this was the plan

[Egor Yakovlev]

Similarly, I was under impression that registry key modes "3" and "5" made it as options (b) and (c)...I doubt we ever had "remove oldest restore point when full" behavior.
Investigating...

[martinic]

It really worked like that, as witnessed from a folder DIR from a backup disk, before upgrading to V12:

Directory of E:\Backups\Backup Job 1_1
18-03-2023 00:14 150.759 Backup Job 1.vbm
04-03-2023 00:53 368.875.663.360 Backup Job 1_1D2023-03-03T234541_8627.vbk
11-03-2023 00:15 175.185.920.000 Backup Job 1_1D2023-03-10T234522_8194.vbk
18-03-2023 00:14 175.178.928.128 Backup Job 1_1D2023-03-17T234516_CFB2.vbk

[Gostev]

The main point here is that there was no work supposed to be done on the retention engine itself, the only change was making these options available directly in the UI. However, the actual code remained the same and as such it should continue working as in V11. If you're seeing a different behavior, then I would recommend copying all existing product logs aside before they get overwritten by V12 logs, as they will help to determine what exactly was happening in V11 and what has changed in V12.

[martinic]

Support Case 05965686 has our logs attached and includes logs from before upgrading.

[martinic]

Similarly, I was under impression that registry key modes "3" and "5" made it as options (b) and (c)...I doubt we ever had "remove oldest restore point when full" behavior.
Investigating...

Reading this again, more info: I believe it was not 'remove oldest restore point when full' but V11 honored the setting 'Retention policy' in the backup job; we have it set to '3 restore points' and that is exactly how many files it kept on the backup disk.

[Egor Yakovlev]

I have went through support case and it looks like registry key have never worked on your installation.

Let me clear it out: v11 registry key "ForceDeleteBackupFiles" set to "3" was made to clear backup job folder on the rotated drive. Clear entirely. Before any other job activity. That is, if you have E:\Backups\Backup Job 1_1\ with several restore points there, entire folder would have been cleared, removing all restore points for that job. This core logic did not change from v11 to v12, just registry key state was moved to the UI with text explanation "Remove all backups for this job". If you are positive that only oldest restore point was removed in v11, clearly registry key wasn't working on v11 server. My bet is that it was a regular backup job retention clearing oldest restore point for you, and then moving on with the creation of a new backup. Please note, that retention was also applied before job start.

However, we have found cases where applying retention on a rotated drive data before job completes was not the best idea (potentially could lead to a retention policy violation), and we have moved retention to be a post-job activity in v12. Guess that is the reason why your RD jobs are now giving "Not enough free space" - there is no space to fit a new restore point, and older one supposed to be deleted after job completes. It has nothing to do with the registry key and relevant RD Cleanup UI options.

Coming back to your scenario: to keep multiple restore points on a rotated drive you do not need the registry key at all - make sure free space and retention are set accordingly - there must be enough space for a new restore point creation and for retention to apply at the end.

Hope that helps!

[martinic]

Hi Egor,

Thank you for your reply.
I will test and experiment with a smaller backup size, select setting a, with retention of 3 restore points and see if B&R will keep 3 restore points or fill up the whole disk with restore points until the disk is full.

- Peter

[martinic]

Unfortunately there are now five backup files on the disk and it will continue until the disk is full and it will throw an error.
This is the situation since I upgraded and why I started the support case.

The main issue/question remains: how do I configure V12 to honor the restore points I set in retention (3 restore points) so that it will remove oldest file(s), like V11 does?

- Peter

[Gostev]

V12 is identical to V11 from this perspective: the job will remove independent restore points immediately after they fall out of the retention policy. By independent I mean that the latter restore points are not dependent on them (for example, the second increment "I" in a "FIIIII" backup chain cannot be removed until the last increment is still under the retention policy).

[martinic]

"the job will remove independent restore points immediately after they fall out of the retention policy"

So... yeah, umm... that is how I need it to work but it isn't doing that.
It exceeds the restore point amount I set and we daily rotate disks so it makes independed files.

[martinic]

Support Case 05965686

Is there any news on this? Are there solutions?

[dk-one]

I agree with the request:

We back up a single weekly backup to a USB disk. There is NO space on the USB disk for TWO backups. So we also need the old backup to be deleted BEFORE the new one is written.

Is there also a registry patch here? The alternative is that I have to MANUALLY delete numerous backups from USB disks.

[mjr.epicfail]

You could also just run a script to delete the backup in the VBR console (via VBR Powershell) and then kick of the job.
In that case you are sure the backup is deleted the correct way, manually deleting them from the USB disk gives an issue with the VBR DB still expecting them to exist.

[ChristineAlexa]

Just thinking out loud,

IIRC, wasn't there a change (to some backup types automatically, to some, manually) between v11 and v12 that changed the retention from

- Number of days
to
- Number of Restore points (essentially, # of partial/full backups)

Is there any chance the setting isn't as you expect because of that change?

[dk-one]

I wanted to delete the last (and only) USB-Backup via scripting within the backup task to USB.

Could someone give me the correct synthax for scripting when the task name is 'USB-Backup' to delete this backup?