snapshot removal for reverse incremental

[YouGotServered]

Matt's post makes sense to me. I don't use "Transform to rollbacks", but why is it that the snapshot is removed after the synthetic full + reverse incremental creation in the reverse incremental scheme? Seems to me like the snapshot should be removed before, for safety's sake. Or heck, why can't we do both at the same time? One's a repository task, one is a proxy task, they both leverage different components, and in a distributed architecture, different hardware / servers as well. I'd ask why we can't do both at the same time for forward incremental as well. Seems odd that we have to do both of these seemingly independent tasks sequentially. Am I missing something?

I always tell the consultants in my company that faster backups are better backups. Not just because it's super cool to have crazy fast backups that process at several GBs a sec (but it totally is), but the less time we spend with open snapshots, the less time we spend with merges, and a failure of any kind with open or merging snapshots is not good - nearly guaranteed data loss in my experience.

[HannesK]

Hello,
I split your question away from veeam-backup-replication-f2/transform-p ... ml#p372783 because it de-rails the topic explaining how reverse incremental works.

but why is it that the snapshot is removed after the synthetic full + reverse incremental creation in the reverse incremental scheme?

can you please explain which backup mode you mean? Because reverse incremental does not have synthetic fulls.

If you look at the animation https://www.veeam.com/kb1933 -> there is no way to remove the snapshot before the backup finishes.

And doing "post-merge" like "transforms every day" requires more performance from the backup storage.

Customers who have issues with VMware snapshots to use backup from storage snapshots. Storage snapshots are really reducing the time that snapshots are open. Transforms are not _the_ solution for that issue.

Best regards,
Hannes

[YouGotServered]

Hey Hannes,
Thanks for the response.

I split your question away from veeam-backup-replication-f2/transform-p ... ml#p372783 because it de-rails the topic explaining how reverse incremental works.

I'm not sure why this was split, I was agreeing with the original poster and was asking why things were done the way the original poster had explained. This seems to be a continuation of that topic. The original topic was not about how reverse incremental works, but was actually how "Transform previous backup chains into rollbacks" works, which is what I was commenting on, and what the title of the post was. I think it would make sense to merge this back into the original post, especially since the original poster had also included the step-by-step order of the workflow I was questioning and commenting on.

can you please explain which backup mode you mean? Because reverse incremental does not have synthetic fulls.

In reverse incremental, the latest restore point is always a full backup. I called it a synthetic full (maybe incorrectly) because of the merge operation, but that's also what Matt called it in his first post and no one said that he was incorrect, so I figured that was the correct terminology.

If you look at the animation https://www.veeam.com/kb1933 -> there is no way to remove the snapshot before the backup finishes.

And doing "post-merge" like "transforms every day" requires more performance from the backup storage.

I think there's come confusion here. I'm talking about the VMware snapshot of the VM we're backing up. Once the incremental backup is taken, why can't we merge the VMWare snapshot of the VM, and THEN merge the incremental into the backup and create a VRB file? Or why can't we merge the VM snapshot and do the backup transform operations together at the same time?

[foggy]

Once the incremental backup is taken, why can't we merge the VMWare snapshot of the VM, and THEN merge the incremental into the backup and create a VRB file?

It doesn't work as an "increment first, merge after" process. Think of that as an on-the-fly blocks replacement in the full backup and writing the replaced blocks into a VRB restore point. Unlike forever forward, where the increment is being created first with the subsequent merge of the oldest increment into the full backup.

[YouGotServered]

Sorry for the slow response.

I guess I'm a little confused, the original post in the topic this was split from (which I still don't understand, especially since that post morphed into a conversation about tape backup) mentioned the process was this:

When the reverse incremental job runs it works like this:
1. Take snapshot of VM
2. Do incremental backup of VM
3. Create synthetic full for today and create the reverse incremental.
4. Remove snapshots.

This post makes it sound like that an incremental job is run, then it's merged into the full and a VRB is created after that, and then the ESXi snapshot is merged.

What you're telling me is that as the backup runs, data is injected directly into the VBK, and simultaneously those older overwritten blocks are written to a VRB. This makes sense, but what I don't understand is why this was not clarified in the original forum thread. Reading through the original thread, it seems like everyone posting (customers and Veeam staff) agrees with the order and process in the original post and no one points out inaccuracies, then my post gets split off into a new thread and then it's clarified separately that the way it was described originally is not accurate.

It just seems inconsistent to me, that's all.

[foggy]

Well, it might indeed sound like that, I admit. In reality, all actions mentioned in pp.2 and 3 are done in parallel, with incremental changes reading starting first and then the synthetic/injection activity. Sorry for the confusion.

[YouGotServered]

Got it, thanks for the clarification. Just wanted to make sure we were all on the same page! Then, going back to the original forum post - I guess given this new information - it doesn't seem like a big deal. Going back to the original topic that this was split from, I guess I don't think this makes sense anymore because it isn't possible, since it all happens in parallel:

If you are going to remove the "Transform previous backup chains into rollbacks", please (if possible) change the logic on reverse incremental to remove the snapshot after the incremental backup is complete and before the synthetic full occurs.

I know reverse incremental requires more I/O, so technically, it is slower, resulting in longer snapshots, but with good backup storage, it should be a negligible difference. If that is a big deal...then use forward incremental

Please let me know if I've misunderstood anything here.

[foggy]

Yes, your understanding is correct now.

[RubinCompServ]

So a Reverse Incremental uses more I/O is because it has to write two sets of blocks instead of one (one to the Full and one to the VRB) as well as an extra read (to read the blocks to be replaced)? In that case, performing a Forward Incremental with daily Transforms would actually require more I/O, because it rewrites the Full, correct?

[Gostev]

That is correct. The primary benefit of this mode has been about being able to move the "rewrite the full" I/O part to the synthetic full day. So with daily transforms, there's no gain - and it's worse than reversed incremental in terms of I/O load. Thanks!

[YouGotServered]

One benefit of reverse incremental is that if someone does a "dumb" and fills up the repository, you can easily delete reverse incremental restore points from the back end of the job to recover some space immediately.

Obviously, this shouldn't happen with good planning, but mistakes happen, and it's hard to get out of this situation with forever forward incremental.

[RubinCompServ]

That's why I've been doing Forward Incrementals with Rollbacks