Is it a good backup plan?

[flibouille]

Our IT service provider has provided us with a backup plan that seems incomplete.

It is a server with a dozen VMware virtual machines (Windows 2019).
- Backup to nas 1 (location 1, local), reverse incremental, no GFS, no AAIP or guest processing, 30 retention points.
- then backup to nas 2 (location 2), idem.

The backup 2 redoes the entire backup, so backup1 + backup2 lasts 10 hours!

Isn't there anything better to do?

Thanks.

[PetrM]

Hello,

First of all, I would recommend to enable guest processing to ensure application-data consistency and smooth restore. Also, I don't see a reason to make backup twice but you can create a backup copy job to transfer backup data from nas 1 to nas 2, it will allow you to follow 3-2-1 rule. Thus, you decrease source infrastructure load and shorten backup duration as only the primary backup job must fit backup window.

Thanks!

[chris.childerhose]

Also, to add to this - would not recommend doing Reverse Incremental backups. Instead, do just Incremental with Active Full unless the repository server is ReFS/XFS then do Synthetic Fulls instead.

[flibouille]

I agree, I do not see the need for 2 backups. So, ok for a backup copy.

Does AAIPP have any drawbacks?

Is it useful to activate GFS for the weekly if you have 30 retention points?

Chris, why you don't recommend reverse incremental? It's slower but I like the idea to always have a full (I don't understand what a incremental with active full could be). Repo are on Synology, btrfs formated.

[PetrM]

Hello,

AAIP does not have any drawbacks It makes sense to activate GFS only if you need to store backups for longer period of time. If the only purpose is to have 30 points for short-term retention, I don't see a reason to enable GFS. If you would like to have just one full backup, you can leverage forever incremental chain as it requires just 2 I/O per data block whereas reversed needs 3 I/O, see more info on this page.

Thanks!

[LickABrick]

Active/Synthetic fulls are made by taking all restore points up until the last full backup and merging them into a full backup. If you have ReFS/XFS it would not take up any extra space (with synthetic fulls atleast). If you don't use ReFS/XFS and are tight on disk space you could use reverse incremental.

Personally the only upside of using reverse incremental that I've found is that if your repository is full you can easily delete the oldest snapshots without breaking your backup chain but of course you have monitoring so this won't happen

More info on Active/Synthetic Full Backups: https://helpcenter.veeam.com/docs/backu ... ml?ver=110

[chris.childerhose]

I agree, I do not see the need for 2 backups. So, ok for a backup copy.

Does AAIPP have any drawbacks?

Is it useful to activate GFS for the weekly if you have 30 retention points?

Chris, why you don't recommend reverse incremental? It's slower but I like the idea to always have a full (I don't understand what a incremental with active full could be). Repo are on Synology, btrfs formated.

Yes, it is good for that but soon it will be deprecated in a future release so you won't have that option. Also, the performance of it as noted is slower with 3 I/Os. Plus doing Incremental with either Synthetic or Active Full is very similar also as noted.

[flibouille]

Thank you for your answers.

If I understood correctly :

• In reverse incremental, I have a always an up-to-date full + as many .vrb as restore points. So for 30 restore points, I have 1 full .vbk + 29 .vib

• In incremental + synthetic full, I have one (or more but let say one) synthetic full per week + 5 .vib. So I have about 4 full and 26 .vib.

• In active full it's the same except that it takes data from VMs and not from the repo.

Is that it? If so, I'll lose a lot of space and won't have enough space to keep this 30 restore points anymore.

And why would Veeam remove reverse incremental?

[william.scholes]

When we backup to NAS, we always try to present the NAS as an iSCSI storage to Windows host so we can format with ReFS.

[infraerik]

Follow up on the iSCSI to ReFS vs SMB. Please avoid using the SMB if you can. It's horribly inefficient and the least secure of the possible setups. If you're using a decently equipped Synology with some extra memory, consider using this approach in order to get a more efficient and better protected repository:

https://infrageeks.com/post/2022-01-27. ... ology-nas/

[ITP-Stan]

Reverse Incremental is very heavy on I/O, many (smaller) NAS devices have very poor performance when you use that.

With plenty back-up space, I would choose NTFS for the primary repo for back-ups. And ReFS for a secondary repo with back-up copy (maybe also GFS).

With limited space for your primary repo, ReFS is a good option with a forever incremental chain with Synthetic fulls (wee normally choose weekly synthetic fulls).
You could even do the synthetic fulls on a daily basis if you really want to have a full VBK with the latest back-up in it (like reverse does).
The synthetic fulls do not take up any real space on ReFS, only the changed blocks.

We do the ReFS over iSCSI to NAS as well, but ...

ReFS over iSCSI to a NAS is not officially supported by Microsoft, so Microsoft says it might break and we won't help.
I'm okay with that for my back-up storage, but something to consider!

The back-up server needs to be Windows Server to create a ReFS volume.


PS: And to meet the 321 rule you should also have a copy off-site, maybe in the public cloud (S3, Blob, ...) or with a Veeam partner in their private cloud.

[ITP-Stan]

Instead of "forever incremental" I meant "forward incremental" in my last post just above this one.

Forever incremental is actually like forward incremental but with only 1 full (the oldest) (so no synthetic or active fulls) and X number of incremental restore points.
In this case each day the incremental that falls out of the number of points gets injected in the full.

Also something to take in to account. When using forward incremental in the forever forward or with synthetic fulls method, it's best to enable periodic health checking in the job.

For a more detailed description see: https://helpcenter.veeam.com/docs/backu ... thods.html

[flibouille]

Thank you all for your precise answers!

We use Synology 920+ with Btrfs formatted SHR (Syno RAID) which sounds pretty good. We have limited in space (currently use 80% of space with daily (30 retentions) + monthly (6 retentions)).

Do you think it's worth destroying everything to switch to ReFS?

For the 3-2-1, I already have a backup on S3 but it's incredibly long. Do you backup daily with forward incremental also in cloud?

[dloseke]

I personally no longer use REFS on a NAS. I have several clients still on it (ISCSI volume to the ESXI host presented to the VM as a RDM disk), but there's a risk there between some of the instability of REFS and the fact that the Synology (also QNAP) NAS will be performing software-based RAID with no battery-backed caching and no controller/network redundancy. I actually am avoiding NAS's altogether anymore and am using a physical dell box with local storage with a battery-backed RAID card, RAID 5 or 6, REFS formatted. I think it'd be a bit safer to use a Linux repository with XFS instead of Windows with REFS as I believe XFS has been proven to be more stable, but I haven't gotten there yet. If you're okay with accepting the risk of REFS on a NAS, then yes, you'll get much better performance and space savings using block-cloning on REFS with synthetic operations, but that's not something I'm willing to risk going forward.

[chris.childerhose]

Just to add yes Linux XFS is very stable and same space savings as ReFS. So a good option for repos.

[RubinCompServ]

Personally the only upside of using reverse incremental that I've found is that if your repository is full you can easily delete the oldest snapshots without breaking your backup chain but of course you have monitoring so this won't happen

I seem to recall Veeam saying (a long time ago) that, because most restores are done from last night's backup, there's a performance increase in always having the Full as the most recent backup so that Veeam doesn't have to assemble all the pieces first.

[dloseke]

I suppose that may be the case, but I'm not sure how relevant it's going to be in real world scenarios. An Instant Recovery is very fast and doesn't matter of the most recent is a full. A restore using only changed blocks won't care if the most recent is a full or not because it still has to go through the individual changed blocks on the target VM being restored. The only case I can really see that on is that is if you're doing a regular full restore, but I'm betting there isn't a huge difference unless you were to have a REALLY long backup chain where it has to read blocks from several incrementals and the full. But on a day-to-day basis, it's going to be a more impact on the backup cycles as it writes in new blocks, reads and writes out the old to the incremental behind it when using Reverse Incrementals..

[RubinCompServ]

Most of our customers require 30 days of retention (we have a couple that do 90 and I mandated a monthly Active Full), which means our options are either:
* Forever Forward Incremental, which will require reassembling up to 30 restore points on multi-TB servers
* Weekly Fulls (Synthetic or Active), which will consume storage like it's going out of style
* Daily Synthetic Fulls with Transform Previous Backup to Rollbacks, which has a tendency to break on the Transformation stage
* Reverse Incrementals, which are slower to run but quicker to restore and more economical

If I've missed something, I'd love to hear about it, because none of these options are overly thrilling to me. Even if there was a option that let us schedule the Full for every 2 weeks (or some other interval) that would be better than a weekly Full.

[LickABrick]

If you have ReFS or XFS your synthetic fulls wont take up any extra space since fast cloning is used. I would be interested to see a restore speed comparison however between a reverse incremental an forward incremental.

[RubinCompServ]

We're currently using ReFS but when customers perform an Active Full, we see their Space Used counter rise considerably.

[LickABrick]

Active fulls don’t take advantage of fast cloning. Also I believe ReFS won’t work if you moved your backup files from something like NTFS to ReFS until the next Active full

[chris.childerhose]

As @LickABrick noted, active fulls don't use space savings on ReFS. Still, I have to ask if you are using Forward Incremental with a Synthetic Full to take advantage of the Fast Cloning, why do they need to take an active full backup as it is "technically" not required? Yes, some users prefer to do this, but you need the space, and when that is a concern, the Synthetic should do the job.

[RubinCompServ]

We moved from ext3 to ReFS by placing both repositories in a SOBR and either letting the jobs perform new Active Fulls or evacuating the old extent. If I'm going to have to run new Active Fulls for all of the backup jobs, this has become the least tenable of the various backup options. As for why some customers take Active Fulls, remember that we're not seeing substantial space savings from the Fast Cloning, many customers are using Reverse Incrementals to keep their Full recent, and, sometimes, the Veeam backup chain for a given server/job breaks for no apparent reason, requiring an Active Full to start a new chain.

[dloseke]

Strange that you're seeing broken chains. I've noted that to be quite common with ShadowProtect, but I haven't seen that with Veeam. What is being used as the backing storage for repository (performance tier if using a SOBR)? I've read about issues with SMB NAS's such as Synology and QNAP creating corruption but the health checks catching it and fixing things up. If this is a common issue, I'd certainly have a ticket opened up with support to figure out why this is happening. If you're not seeing much in space savings due to fast-cloning, I'm guessing the change rate may be really high on these machines? Otherwise it seems strange to me.

[dips]

One of the main reasons I found when the backup chain breaks is due to the backup process being aborted. For example, an inadvertent reboot of the backup server or a crash of the Veeam services on the backup proxy but it is quite rare.

[RubinCompServ]

Strange that you're seeing broken chains. I've noted that to be quite common with ShadowProtect, but I haven't seen that with Veeam. What is being used as the backing storage for repository (performance tier if using a SOBR)? I've read about issues with SMB NAS's such as Synology and QNAP creating corruption but the health checks catching it and fixing things up. If this is a common issue, I'd certainly have a ticket opened up with support to figure out why this is happening. If you're not seeing much in space savings due to fast-cloning, I'm guessing the change rate may be really high on these machines? Otherwise it seems strange to me.

We're using a NetApp E-series as the repo. One of the more common errors we're seeing is, "Unable to find parent by original parent id for oib". When I worked with Veeam support on this, the end result boiled down to, "I don't know why it happened and there's no way to repair it; You should run an Active Full to create a new chain."