Phasing out Reverse Incremental?

[APangF1]

Has there been discussion about phasing-out Reverse incremental? I saw a discussion where Gostev chimed in about this, but cannot find the posting again.

Reverse incremental is great when sending TB sized backups to tape, without having to create a synthetic full (uses more disk space) OR a virtual full (which incurs a heavy resource load on the backup repository server)

Keep reverse incremental for Hyper-V !!!

[foggy]

Hi Albert, based on the usage stats and QC overhead we're phasing out the transform into rollbacks option. It will be hidden in v11 for the new product installations, and the plan is to completely remove it in v12. That said, reverse incremental mode will continue to exist.

[Gostev]

Here's our discussion about reverse incremental earlier this month. This basically explains that there are no use cases left for reversed incremental, specifically around tape. If you truly worry about the repository load, then you should definitely stop using reverse incremental mode regardless of whether it will remain (your perception of a virtual full is no longer correct as of v10). Thanks!

[APangF1]

How is virtual full to tape improved on v10?

The reverse incremental option will be hidden in v11, but the mode will still exist ?!?

Thanks

[Gostev]

It was improved drastically... became like 4-5 times faster while reducing the overall storage load. In v11, the same tech will be applied to all reads from backup repositories (so also restores, backup copies, etc.)

Again, as foggy already stated above, nothing at all changes in v11 with the reverse incremental option. There's just no good reasons to use it any longer, at least for the reasons you stated.

[APangF1]

Okay, sounds like reverse incremental is hidden as an option for new jobs in v10+, but still works for existing jobs from v9.5U4.

Is there a document or article about the way virtual full to tape is improved in v10+ ?
My understanding is that in v9.5U4, a virtual pointer file is created in the backup repository server’s memory/virtual memory, prior to writing to tape.

Thanks

[Gostev]

Just to be clear, the reverse incremental option is NOT hidden for ANY jobs either in v10 or v11. And there are no changes planned around the reverse incremental option in the foreseeable future.

We don't document low-level implementation details, since it also constitutes know-how. But no, even 9.5U4 never had to create any special "virtual pointer files". If you think about this, copying certain virtual full state to tape does not require anything different comparing to performing a restore from the same restore point. In either case, Veeam needs to address directly the specific "virtual full" state of the selected restore point, which is comprised of data blocks sitting in various full and incremental backup files. So this has been in the product since v1, and is extremely polished already. For example, you probably already noticed that the restore performance is the same whether you restore from the latest restore point (which is stored in a single VBK file in case of the reversed incremental backup mode), or from earlier restore points (which in turn requires restoring from the "virtual full" representing an earlier image state).

[nunciate]

It looks like there is still some base issue with GFS media pools and forward incremental. I tested a long time ago and realized that full backups were being sent to the daily retention of a GFS tape job. It looks like this was fixed for backup jobs using a weekly transform but not for regular forward incremental. When you do forward incremental it eventually starts rolling the oldest daily into the VBK file and then that full backup is sent to tape under the daily retention. That is not good. I rely on the ability to send only VBR daily incremental to the daily pool and only full VBK files to the Weekly/Monthly pools once a week/month. I'd love to remove the weekly roll-up (not reverse) as it does add a load to my server and it also eats up a lot of space having all of those VBK files. Even with REFS and Dedupe, I have to closely manage my storage as it will fill up. Doing forever incremental means I only need to maintain the single VBK but as expected it wants to tape that full backup out every day. I don't have enough tape drives to allow for a full backup to tape for every job every night. For now, I am sticking with Daily incremental with a weekly roll-up. Keeping my standard tape out jobs and running scripts once a month to move the jobs from Weekly to Monthly and then back.

[joachimeberhard]

I am using Reverse Incremental for our required daily full backup to tape.
It is our primary usecase of Veeam and the GFS Archive Foward Incremental and the Replication Features are just bonuses on top.
That being said, reverse incremental cuts down the main backup job time to about 1/3 compared to active full.

In my humble opinion, tape backup features of Veeam are very superiour to other products on the market.
It is why you won us as a customer.
There are many competitors when it comes to VM backup or Cloud backup, but imho you flat out rule when it comes to tape support.

[nunciate]

I agree Veeam is far superior to everything else out there. I was using reverse incremental for a long time and switched for forward incremental after purchasing a new repository server with a lot of space. Apparently not enough space for that type of job though. Unfortunately, since moving the jobs away from reverse incremental I now have no option to go back. You have no options to choose what day to do the roll-up with reverse on new jobs and disappears from old jobs when you switch. So it is effectively the same as doing a forever forward where the oldest backup is pushed into the full every day.

[joachimeberhard]

Well, I "fixed" this issue by having two repositories and two jobs, one for the GFS Archive which will eventually become huge, and one as a "working" repo for the reverse incremental and the physical machines and other backups with a simpler retention.

Imho both things, reverse and forward incrementals have unique and important traits to us that justify both.

But from our CEO and CFOs perspective, its the "daily FULL on tape in the safe" that makes them sleep well.

[joachimeberhard]

And since we do tape only on weekdays, i can do a active full for the reverse on fridays and the active full for the gfs on saturday.
Since the forward incremental is very fast, I can just chain those job schedules together and its done.
They never get in the way of each other, I think its an elegant solution.
And it also complies with the 3-2-1 rule.

[nunciate]

We have about 250 VM that we backup using about 134Tb total. We currently set our forward incremental to keep 21 retention points. I want to keep 28 days but with the way backups fall off that was filling my storage as the oldest full wasn't dropping off for almost a week after the most recent. Our repository server has about 280Tb usable space and I currently have 32Tb free.

[joachimeberhard]

I kinda envy you for having to work with such a large environment with latest backup hardware and ReFS, I guess on SSD.

My usecase is much smaller with 2 entry level nas with about 100TB of cheap storage.

But it works excellent when consistently using 10Gbit network.

Veeam is perfect for small industry, and I hope they stay this way.

[nunciate]

I use HP Apollo High-Density storage servers. Production has 12Tb 7200rpm disks but I have about 12 SSD used for HPE Smartcache in there as well. That helps increase performance without breaking the bank on all SSD arrays. I have a similar setup in our DR facility with 6Tb drives I have double the drives in DR as Prod so usable space is close to the same. Everything is connected on the backend with fiber channel so we don't have to go over the network which is only gigabit here. The tape library is connected to the storage via FC as well. The setup works quite well. Our owner requires overkill on our backups so we back up every VM to disk(We are 99.99% virtual). We then tape out each night. Prod backup jobs are chained to replication jobs and every VM is replicated to DR each night. Replication jobs are chained to other backup jobs which backup all the replica and other live VM in DR to disk there and then those are also taped out each night... Like I said, OverKill but we are 100% protected in both locations on every single bit of our data.

[joachimeberhard]

We kinda went the opposite way and went all in on 10Gbit networking, since there are now very cheap NAS systems that can easily deliver 10Gbit even with Sata disks and also very cheap all 10Gbit switches.
Of course we can't use deduping filesystems on this.
We used this opportunity and added several 2port 10Gbit cards to the servers for the Virtual Machines, and we got a very massive daily performance boost on production, while still using only 10K HDDs all around.
For the little investment compared to high-end storage, we were surprised how big the performance boost was in daily use.

EDIT: For the distant future I imagine something like HPE Simplivity HCI cluster nodes, all NVMe. I hope this dream becomes reality one day.

[nunciate]

I have been digging a bit more because I have a lot of storage but for some reason, it just keeps filling up and I couldn't figure out why. Well, I think I know now. I have all this storage broken up into 5 volumes/repositories of about 50Tb each. I needed to keep them below the 64Tb limit on deduplicated REFS volumes. So I have all those included in a single Scale-Out repository. After investigating further I found that Veeam was writing VBK for jobs onto different repositories. I'd expect it to put most VBR on one repository for a single job and all VBK on the same volume. Since it isn't doing that the O/S doesn't have an opportunity to duplicate out the additional space from all the duplicate VBK and so my volumes fill up. That is Bad and I am not sure how to fix it.