[Feature request] Delete oldest replica at beginning of job

[v.tom]

This is a feature request from a customer of us:

A Job with 100 VMs should keep 5 recover points.
During replication, Veeam first create a temporary 6th recover point. This costs a lot of unnecessary space. In our case we have a vm of 1.1 TB with a change rate of 150GB-250GB data per day.
The total change rate and simultaneously running jobs need a lot of extra temporary free space.

The feature request is about the possibility (in case of storing multiple recover points per vm) of first removing the oldest recover point at the beginning of the job. This will free up the space, after that the new recover point can consume this space.
Result: jobs that work AND you may be able to keep more replica's (continuity AND value).

We hope that this can become available as an extra option to configure. Ofcourse the worst case scenario is, if the replication job is failing, then the oldest
recover point is already deleted at the start of the job.
In our case this is acceptable behavior.
But as this would be an option, it gives you the possibility to decide if you want this or not.

[Shestakov]

Hello Tom,
Running datastore with a lack of space that replication job can`t create another restore point is not the best practice.
And as you said each time the job is failed one restore point is to be deleted. That also goes against basic data protection principles.
Have you considered decreasing the number of restore points to 4?
Thanks!

[v.tom]

Yes this has been considered. But the customer tries to optimally use the disk space available.
The customer response was:

Within Veeam B & R we would like to be able to choose if the oldest replica is being removed. So far this is only at the end of the job, but in our situation I would like to choose this at the start.

Assuming that normally the replication will be run successfully, this would be a better solution.
-(under normal conditions) The replication will be successful as the space is guaranteed.
-It might be even possible to make one more extra restore point as space doesn’t have to be double reserved.

In the current situation I have a failed replication and it takes me lot of effort to fix it (time, effort)
-Some of the replica VMs have to be deleted to create space. So a job can run successfully again.
- Or make new jobs which is really not convenient
In Short: a lot of effort to make space. And worst case if I need to do a failover, I have to use an older replica then desired.

In the proposed situation, I take for granted that I lost the oldest restore point, but this is also the case if the Job was completed successfully. In the Job settings it only have to be a checkbox, so the customer can decide.


My personal notes:

I think our customer has a point.
There a mainly two reasons why you want to use replication:
1. As a glorified backup, with the benefit you can startup the machine very quickly if the primary fails
2. To ensure you have a recent copy of your production VM's on a DR site.

In scenario 2 I would prefer a more recent replica, instead of the "safety" of not deleting the oldest replica.

[foggy]

I hear your arguments, however, what Nikita was trying to say was that running in a space constraint situation is not the best practice at all. VM data tends to grow with time thus requiring more space on the target datastore, so you will finally have to extend it anyway. Also, think of the situation when some huge changes occur inside the replicated VM that would require much more space on the datastore than the oldest restore point consume.

[ssimakov]

Just imagine vacation period or Christmas holidays (or any other long holidays for that matter) when there is no close supervision over backups. Deleting oldest restore point before new one was successfully created is terrible idea as you might end up with no replica restore points at all - all of them will be deleted in case replication starts failing during that period.

[Shestakov]

Tom,
your point is clear and may look like a workaround for your customer, but it`s not to be implemented as a new feature because of the reasons mentioned above.
We could make the feature smart enough not to delete all restore points if the job fails or performs retries, but it would be useless in cases, such as Foggy described, when new restore point is much bigger than deleted oldest one and you can`t fit it because of the space shortage. This case is to happen often because only customers who are short in space would want replication work this way.
I would suggest your customer either to increase/re-arrange datastore space or to decrease the number of restore points if first is not an option.
Thanks!