Master Veeam Tricks Volume 1

[Cragdoo]

Happy to share a Veeam Vanguard community project that was started on the back of the Veeam Vanguard summit in Prague, 15th-18th Oct ... This is hopefully the start of 2-3 volumes, feel free to download and spread the word. It's basically a collection of blog posts and tips from Vanguards. Huge thanks to Dave and Cristal Kawula for pulling this all together and the other contributors. We hope you find this useful.

https://leanpub.com/masterveeamtricksvolume1

**Note to admins** apologies if this isn't the place to post, was just too good not to share.

[DonZoomik]

While browsing free sample, there's talk of Win 2019 with ReFS deduplication. Previously ReFS block clones and deduplication could not be used at the same time. Has this changed? Otherwise there would be little advantage over NTFS with deduplication...?

[Gostev]

Update 4 should have experimental support (behind a registry key).

Until then, you should configure Windows deduplication to only process files that are not subject to future block cloning operations - for example, sealed GFS full backups. To achieve that, you just configure Windows to deduplicate files older than X days, where X depends on your GFS frequency. You will for sure run into issues with the default setting (5 days) since it's not enough to cover even 1 weekly backup.

[DonZoomik]

Block clones with dedupe integration? Now that'd be really cool! Sounds almost like StoreOnce Catalyst or Data Domain Boost.
I presume block clones the would work directly on chunk store metadata. I'd really like some background details.

I haven't pinged deduplication team yet nor tested myself but has 4TB file size limitation been increased?

[Gostev]

Nope, no changes there with Windows dedupe, still all the same file and volume size limitations... so per-VM chains is a must.

Yes, it should work directly with the chunk store. We've just implemented everything that was required for the integration, but we won't really have a chance to run it through its paces until after we push Update 4 out.

[orb]

Dedup over ReFS has a use case. I did not try Windows 2019 and my average feeling with Windows like it mentioned above is the filesize/lock limitation. The lock is a real pita ! you find out often your dedupe process started for a while and windows aborts it for some locking reason and without having accessing it. It always means starting over again.

My experience ise with dedup level (e.g. NetApp). I manage to have between 18%-25% gain on a ReFS volume and no uncompress option enabled.

Oli

[DonZoomik]

In my experience "starting over (from beginning)" in case of locking is a myth. Yes, deduplication aborts file processing if there's an exclusive lock on file but it doesn't restart from the beginning on next job run. It just continues from where it left off. For example, when a 100G was 80G processed and something locked it, deduplication would process continue after 80G point and just continue to process the remaining 20G.
You can indirecty see file(/reparse point) shrink (size on disk) during processing. When aborted, it doesn't grow back. When next job starts processing, it continues where it left off.

Also, local locks (eg locked by process within operating system) are "harder" than remote (for example SMB). I've seen that remote locked files are processed successfully but local locked are skipped. I haven't checked handle/lock types for details, just an observation. Maybe that's why MS recommends separation of storage/deduplication and HyperV on different servers/clusters.

[DonZoomik]

Yes, it should work directly with the chunk store. We've just implemented everything that was required for the integration, but we won't really have a chance to run it through its paces until after we push Update 4 out.

Would it be possible to test it (if U4 is released within a few months)? We have a project coming up in a few months that really could make use of it.

[foggy]

Yes, you will be able to enable and test it once U4 is out.