Forever Forward Incremental - Time of Full Inject

[LarsWulf]

In the past we used Reversed Incremental as we also had quite good storage performance but less space. And we did quite well with this feaure. Now I noticed the new Forever Forward Incremental which is great. And I do understand the use and benefit of it.

But there is one thing that is not clear to me and where I have an idea:

Lets assume I have 10 jobs configured with Forever Forward Incremental with the same Repository and the Runtime is very different. So the first 3 jobs might be finished with incremental backup quite fast while the other jobs are still doing incremental backups. So I assume that these backup job will start to inject the oldest incremental Restore Point into the Full Backup immediately like it happens with the Synthetic Fulls.
Is this assumption correct ?

If yes the storage load would increased because the injection of the incremental into the full takes doubles the I/Os compared to just a simple Incremental. And we have for these jobs more or less 50% random read I/Os instead of only sequential writes. So this might slow down the whole backup process also of the backups that are actively doing their incremental backups and producing load on the production storage.

How about the idea to define the point in time where the Inject into the Full Backup File happens. In my scenario the perfect time would be after all 10 jobs have finished their Incremental Backups. Because the backup itself is finished and only the "cleanup" on the Repository needs to be done. So spoken you can split the Backup Job into two parts:
1) The real backup where you have increased load on your production storage
2) Cleanup Phase where you only have load on your backup systems but no additional load on your production storage

I hope the idea is clear. And if this idea is not possible or their would be no real use please let me know.

[dellock6]

Hi Lars:
- correct, injection is executed immediately after the new incremental is created
- correct again, the overall I/O of the repository should take into account the injection activity of jobs that already finished the incremental part. But note that this is again lower I/O than reversed, since the incremental run (that is 1/3 of the overall IO of a job) is sequential, while on reversed is almost entirely random, thus with a higher impact

Yes, your idea is clear, we can explain it as a deferred injection activity. In a way it would make sense since the new incremental point has already been created, so there is no violation of the retention lenght in terms of missing points (at most, there would be for some time an additional restore point that needs to be injected).
Another way would be, instead of waiting for jobs to finish or complex scheduling, to simply check if there is no backup job occurring into the same repository before starting the injection. Be aware however that once started it would not be interrupted to avoid inconsistencies, so if another backup jobs starts after a while, there would still be the injection I/O "interferring" with the backup I/O.
And also, with your design, I/O will be impacted even more because the injection activities will happen all at the same time, instead of being distributed over time.

I think the best solution is still to design the repository from the beginning to be able to cope with the injection I/O of all the jobs, instead of simply planning it for the incremental runs.

[LarsWulf]

Hi Luca,

thanks for your detailed reply.
All your mentioned points are correct and I am aware of these.

And of course you should normally have an repository that can handle the injects. But in reality I am responsible for the technical design of a development only environment. And as you might imagine, the budget for backup is not always very big. But I have to backup around 1300 VMs with around 120 TB of live data. So I have to find an optimal way of doing this with less money as possible.

The basic idea behind my suggestion is not to only to have slower backup storage it is having the shortest possible time window where the productive storage is facing additional load produced by the backup jobs.

In my case it is top priority to have a backup but with only very less impact on the production storage.
The RTO is not even really defined (unfortunately). There is not even a DR site. From daily operations point we normally have 2 or 3 restore requests per month. And it doesn`t matter if they take 1 or 4 hours.

So if your described scenario (inject checks for other running backup job into the same repository) might be an additional advanced feature for future Veeam Releases. It would give me and other users even more flexibility in their backup design.

[dellock6]

Well, we introduced in v8 Backup I/O control for production storage, maybe the same logic can be introduced for repository too. Let's see what will come, but thanks for posting here the request, as we always say every request counts.

Luca.