ReFS aligned compressed backup to tape - keep tape compression enabled?

[DonZoomik]

While it's recommended to keep tape compression off (already compressed by Veeam), I think it might be beneficial if ReFS is used.
Alignment required for block clones wastes some disk space - I haven't measured hard numbers but I've seen some estimates of up to 10%.
Alignment padding should be essentially blocks of zeroes that should compress very well in tape drive as tape drive has no need to keep alignment. This should effectively gain some space wins on tape - fitting maybe an extra TiB on LTO8 would be great.

Is this reasoning correct?

[Dima P.]

Hello DonZoomik,

Can you please clarify if your device supports hardware compression or you are referring to software compression? Thanks!

[DonZoomik]

My tape drive supports hardware compression and my Veeam backups are already (software) compressed.

An example (incremental but likely the same for full backups) - I have a job that shows transferred size of 619MB, however on-disk files are 717MB - presumably due to ReFS alignment adding in zero padding. If I wrote it to tape and kept hardware compression enabled, I would get most of this 717-619=98MB back as savings -right?
From what I've found, practice of disabling tape compression dates back to pre-ReFS (when non-alignment would have likely meant 619MB on-disk file or very near that). Or is there anything I'm missing?

[veremin]

Not sure whether I follow the logic behind the provided formula, but yes, the backups stored on local repository is likely to occupy less space on tapes due to hardware compression. Thanks!

[DonZoomik]

Several years later finally got a chance to test out the theory.
When source is an aligned repository (ReFS, XFS), I can get consistent 2-6% (avg ~4-5%) compression (after about 20 LTO8 tapes), presumably from zero padding. Not much but a few hundred GB per tape can be a bonus.
I'd say that on LTO6+ and SLDC it'd be worth to change the default recommendation. Drive hardware compression is constant time and essentially free so there seems to be no downside to enabling it.

[HannesK]

Hello,
thanks for providing the test results.

I will talk to my colleagues from the best practices guide team about it.

Best regards,
Hannes

[blithespirit]

Thanks for this clarification, we will update the BP guide to reflect the findings, I am glad you finally got the chance to test it and lay your theory to rest!

[DonZoomik]

The compression efficiency is still largely YMMV affair and depends on how much padding there is. Did some back-of-envelope calculations.
For the sake of argument, let's assume 1MB block size for reference. I haven't found out if tape block size affects compression window but it should have little effect as zeroes compress well either way and I didn't see obvious difference between 256K and 1M block sizes, maybe the drive has some internal fixed size.
For example on 64kB cluster ReFS with 2:1 Veeam compression ratio (just as an example that you could fit 2 Veeam storage blocks within 1MB), you could theoretically see up to (1024kB - 128kB + 2B) stored with near 128kB ( ~12%) wasted. In real life it's of course much lower, depending on how much padding there is. It's statistically probably half that (64kB, ~6%) and tape drives constant time hardware compression is also likely not very efficient so you could end up with what I'm seeing - around 4%. With higher Veeam side compression you could fit more blocks with potentially more padding.
On 4kB XFS (and 4kB ReFS), potential savings are much smaller. I don't have a XFS test repo to use with this library but potential savings would be max ~1% for the same scenarios as above as the maximum padding size is 16 times smaller.
Still as it is inline fixed function constant time hardware compression, you lose nothing by enabling it, presuming that on LTO6+, uncompressible data is written out as-is.

[HannesK]

yes, XFS or REFS 4KB vs REFS 64KB has impact on the "used space". Just by chance I have a test running for that since around 3 weeks and the difference so far is constantly around 5% less used space on 4KB XFS vs. 64KB REFS