Feature Request: Retention by Destination Usage

[mkaec]

Windows 7 backup does not have retention settings. Instead, you give it a disk and it uses the entire disk. I'd like to see something similar in VEB. Right now, I have to guess how many restore points the destination can handle. I'd like VEB to manage that for me.

If there was an option for VEB to work like Windows 7 (use all available disk and purge old restore points as necessary), that would be great. But, I'm thinking perhaps minimum/maximum settings would work better. This would allow the application to alert when there is not enough space for the minimum restore points (probably how the application works today), but allow for inclusion of additional optional restore points instead of wasting space on a dedicated backup disk.

[Dima P.]

Hello mkaec,

This would allow the application to alert when there is not enough space for the minimum restore points (probably how the application works today), but allow for inclusion of additional optional restore points instead of wasting space on a dedicated backup disk.

Imagine there is no free disk space available – which restore point should be purged? Old one or new one?

I have to guess how many restore points the destination can handle.

Its hard to predict – you never know if the data in the backup can be compressed/deduped well… Additionally, it heavily depends on change rate which is related to the OS activity and such, which again is hard to predict.

[mkaec]

Imagine there is no free disk space available – which restore point should be purged? Old one or new one?

Purge starting with the the oldest restore point. Restore points should be FIFO, just like now. The only difference is the number to retain would not be fixed.

In the case of providing a Min/Max option, if the disk is full and deleting another restore point would go below the minimum, handle it the same as an "out of disk space" error is handled now (probably fail and log an error).

[Dima P.]

Purge starting with the the oldest restore point.

I am worried that with such option retention can go out of control – someday you want to restore the file from old backup and relies that it was purged automatically due the storage limitation you set while ago. Also, I am thinking of potentials issues which might be caused by huge file (like video recording) accidentally placed inside such backup location provoking the backup solution to delete the excising restore points in order to clear some space.

[mkaec]

How is this any different from how VEB works now? If I set 14 restore points in the current release and I need something from the 15th restore point, I'm out of luck.

If you were to add Min/Max, then I could set min to 14 restore points and max to 120 restore points. This would allow me to better utilize storage while maintaining a minimum amount of protection.

And to the concern about a large video file accidentally being placed in the backup location, implementing the backup to an unmounted volume request would help prevent that.

[Dima P.]

If I set 14 restore points in the current release and I need something from the 15th restore point, I'm out of luck.

In Veeam Endpoint Backup we count days when the backup was produced. Eventually, setting the 14 days retention does not mean you will end up with 14 restore points - daily schedule can be combined with any backup trigger and you can create more than one restore point per day.

[mkaec]

In Veeam Endpoint Backup we count days when the backup was produced. Eventually, setting the 14 days retention does not mean you will end up with 14 restore points - daily schedule can be combined with any backup trigger and you can create more than one restore point per day.

My apologies for the incorrect terminology. For purposes of this discussion "restore point" can be replaced with "day" in my text.

How is this any different from how VEB works now? If I set 14 [days] in the current release and I need something from the 15th [day], I'm out of luck.

If you were to add Min/Max, then I could set min to 14 [days] and max to 120 [days]. This would allow me to better utilize storage while maintaining a minimum amount of protection.

[Dima P.]

The only difference with the existing behavior and proposed – the restore point is not deleted when there is no space left on the repository.

[mkaec]

The only difference with the existing behavior and proposed – the restore point is not deleted when there is no space left on the repository.

Existing behavior is that the number of days of restore points to retain is fixed to a single value. Requested behavior is that a range be allowed to be set.

[Dima P.]

Marc,
I am going to take your feature request to next version improvements discussions with the team, so thanks for clarifying a few things for me. Cheers!

[mkaec]

Thanks, Dima.

[kewnev]

+100 on this feature request.
I had wished for this feature several times in the past. kudos the mkaec for requesting it.

Some may prefer having a set no. of restore points, but others like myself prefer as many as can safely fit. It would take out the "guessing game" for the admin of working out an arbitrary estimation on how much backups will fit.
Also if one were to add storage to the repository's array, then Veeam will notice the extra space and add more backups, without the admin having to manually increase restore point numbers.

EDIT:
I just noticed the user was talking about VEB ... but would be nice if B&R could do this too!

[Dima P.]

kewnev,

I just noticed the user was talking about VEB ... but would be nice if B&R could do this too!

Thanks! I'll share this idea with VBR team as well.

[Gerald]

Thank you for making Veeam Endpoint Backup. There is a lot about it that I like (free, simple, good performance)

But Retention by Destination Usage is a really important missing feature. The way VEB currently handles free space is abysmal.

To test VEB, I've created a 60GB (C:) Virtual Machine with a 120GB Backup Drive as a fixed secondary disk.
I've then configured VEB to make a Volume Backup auf the drimary disk to the local storage: my 120GB secondary disk.

I then started to change large data files on C: and making backups after each change. And of course it quickly failed after creating only a few incremental restore points.

Link: http://nv-systems.net/pics/forum/Veeam- ... ckup-1.PNG

It's quite colorful, but not very telling. Nowhere can you see what the last successful backup was! (Hit: It's the yellow one! But you could not even see the yellow one if 7 subsequent ones had failed because there is no way to scroll or enlarge the window)

And without knowing the program, you would think that the top line of text "Recovery media has not been created and 1 other warning" would be important. Actually, it is not. Even if you click "Details" you will not even be shown that several backup attempts (the red ones) have failed! Instead the stupid recovery media message is shown as if that were important...

I would like to make the following suggestions to make VEB and product that you can trust with your data:

a) Please change the wording "keep restore points for the last [x] days when the computer was used". As explained in the forums this does not keep the last days but the last number of backups. The wording is very confusing and also just plain wrong.

b) allow specifying a maximum backup size during backup configuration and remove old backups accordingly. Acronis True Image for example can do this. I always configure my Acronis True Image Backups to "1 Full Backup, then 15 incrementals and limit disk space to 150GB and backup to \\NAS-DEVICE\SHARE\PC-NAME". This works very well, even though it probably needs a little bit more space temporarily when creating the archive (I think). But I can configure it to back up a 240GB SSD to an internal 500GB HDD with for example a maximum usage limit of 400GB and I have set up a perfectly working backup that can pretty much never fail and requires no user intervention no matter how much data changes on the SSD.

c) Actually NOTIFY the user if something is wrong with his backup plan. Currently VEB does absolutely everything in its power to NOT make the user notice. There is NO notification bubble in the tray. There is nothing in the Windows 10 message center about the problem. There is no popup on the desktop as the backup fails. If there is a major problem (and a failed backup is a major problem) VEB should fully open and tell the user. The only thing currently showing a problem is a few pixels on the very small tray icon, which might even be hidden in the tray bar by windows in many cases!

d) When opening the application you should cleanly show if the backup is working / worked the last time. The current way with the color bars is not horrible, but not great either. Please show your programmers / UI designers the way Acronis True Image 2014 (not the horrible 2015) does it.
Link: http://nv-systems.net/pics/forum/Veeam- ... TI2014.PNG

[Dima P.]

Hello Gerald,
You are welcome. Thanks for a perfect feedback!

(Hit: It's the yellow one! But you could not even see the yellow one if 7 subsequent ones had failed because there is no way to scroll or enlarge the window)

I have to say that the predefined schedule is set to run on a daily basis, so in typical daily usage would mean you have a single run per day, so 7 bars on the chart is expected at this point.

And without knowing the program, you would think that the top line of text "Recovery media has not been created and 1 other warning" would be important. Actually, it is not. Even if you click "Details" you will not even be shown that several backup attempts (the red ones) have failed! Instead the stupid recovery media message is shown as if that were important...

The notification breadcrumb shows only “application notifications” not related to backup sessions at all. It displays messages like “Recovery media has not been created” or “New software update is available” while backup job messages are displayed under the bars in the chart a color coding indicates was the job run successful or not.

As explained in the forums this does not keep the last days but the last number of backups.

Can you share the link to the said explanation?

then 15 incrementals and limit disk space to 150GB

May I ask what happens if it reaches the limit? Does it delete the oldest restore point or simply fail?

There is NO notification bubble in the tray. There is nothing in the Windows 10 message center about the problem. There is no popup on the desktop as the backup fails.

Most probably, we are not going to implement pop ups and stuff like that (its very annoying and you usually end up with ignoring such annoying notifications), however, there is always a room for improvements in this field. Windows Action center sounds like a perfect feature request for me

The current way with the color bars is not horrible, but not great either.

Not sure I follow. If the bar is green - you have a successful backup chain and what’s more important the whole backup chain is restorable. What additional info you want to be displayed? Thank you!

[voyager529]

I am weird. I've got 2.5TBytes of storage on my laptop, and I don't have much room left...basically anywhere. Typically though, I back up my C: drive to my S: drive.

What I'd like to do is one of two things. First, I'd like to be able to have an option that says, "if today's backup job won't fit, delete old stuff until it does." Perhaps with an optional pop-up notification that would say "you have backups between 01 Oct and 18 Oct. To perform a backup for 19 Oct, you'll have to delete the following backups: 01 Oct, 02 Oct, 03 Oct. Would you like to skip this backup run, or delete previous backups to make space?".

Simiarly, I'd love to be able to have a "Vote old backups off the island" option. Suppose that on my 1TB drive, I've got 100GB left. It's enough to run the next 55GB backup, but what I really need right now is to be able to create a 250GB VM. I'd like to be able to select backups to delete in such a way that Veeam would be aware of it and consolidate where possible. Ideally, it'd allow me to pick backups to remove, and then have a running total of how much space would be freed up by deleting them, so that I could work accordingly. Perhaps there could be a quick and dirty "delete everything except enough data to turn the most recent successful backup into a bare metal recovery point" button.

Thanks for considering these!

Joey

[Dima P.]

Hello Joey,

Thanks - that is a good feedback. I've merged your post to existing thread to keep the similar requests in one thread. Cheers!

[john.willis]

It would be useful to have a checkbox to allow an Override of Backups to retain: based on available free space percentage of total available on the target.

Just leave the problem of deciding when to trigger an Override up to us.. I understand estimating how much is required for a forward incremental may be difficult.. but if we have a percentage.. it makes it independent of the actual drive or target size.

To be sure, always retain the last Full backup, but bring them all forward so we have at least the latest available until we ran out of space for incrementals.

[Dima P.]

Hi John,

Thanks thats a valid request! The only thing that bothers me in the request is this: imagine you got a large incremental backup file, let’s say you copied HD movie to the local hard drive and then backup kicks in. With the described logic it should overwrite the valid restore point. When it comes to recovery needed restore point might simply be erased.

[john.willis]

I am not sure I understand what your suggesting.

In this theorectical failure point, integrating the old restore point with the new incremental would fail?

I guess your proposing that the last incremental failed, and that bringing the old restore point forward by integrating that with the old (failed) restore point would fail the integration and corrupt the only available restore point.

That is useful information; it tells me the order in the procedure when a new incremental is taken is to [first] perform the step of bringing the old recovery point up to date. And it tells me that there is no check for the validity of the last incremental, or check that the last incremental succeeded.. I would suggest that be a [very high] priority feature going forward for many reasons (a user could change the number of retore points to save in the config and wipe out their backups!).. but if it isn't there now, then your point is valid and understood.

[Dima P.]

Thanks! I was talking about potential issues such ‘backup threshold’ overlap may bring. By the way there is a built in health check even now

[mkaec]

I think this thread may be a candidate for merging with Feature Request: Retention by Destination Usage.

I think this is my #1 feature request for VEB. On endpoints, I've been using Windows Backup. It fills up the disk and then starts culling old restore points as needed to continue on forever. VEB requires setting the exact number of restore points to keep. That means that either I underestimate, and waste space on the destination disk, or have a future backup fail due to lack of available disk space. It would be so much more convenient if I could configure VEB with two numbers for restore points: Min and Max. If the disk fills up, old restore points would be removed to accommodate. If the disk was full at the Min level, then it would fail.

I think this makes a lot of sense for endpoints with dedicated backup storage, but not so much when backing up to a VBR repository or SMB folder. So, I can see what the current design is based on. When the storage is not dedicated to a single system, you want a rigid retention setting. But VEB is great for home users and other systems that do have dedicated backup storage.

[Dima P.]

Marc,

Agree, this feature makes total sense from end-user perspective if you have a dedicated storage per endpoint.

P.S. Thanks for pointing to the correct thread – all posts are now merged in a single thread.

[mkaec]

I'm still really interested in seeing VAW have the capability to set a variable min/max retention such that if the disk fills up retention will be reduced as needed (never going below min) to recover space for the next backup. This is similar to how Windows Server Backup works and it allows for very easy configuration. I don't have to guess how many restore points will fit on the disk. Everything just automagically works. If data size increases, then things automatically adjust on their own. This would be the perfect setting for an endpoint that has its own dedicated backup disk.