SOBR Data Locality with ReFS, XFS

[ferrus]

Is there a way of enforcing a stricter data locality when using SOBR and ReFS?

We have a three 55TB node SOBR, each with a 6-8TB free space margin when fully loaded.
Because each backup job isn't completely even and data amounts change - we've never been able to keep the free space on each node exactly the same, but it rarely differs by much more than 2-3TB.

Recently, I've been reading more an more on the forums here, that synthetic fulls should really be enabled for ReFS repositories - as they are effectively free and increase efficiency by lowering the backup chain length.
I think I may have encountered this situation before and returned to Forever Incremental, but within 48 hours of enabling synthetic fulls on the SOBR jobs - the data locality rules have been broken, full backups have been introduced on other nodes and the free space has been wiped out because of the duplication of backup files across nodes.

Veeam may not see this as breaking data locality as it hasn't split the backup chain (vbks and vibs) across the nodes, but creating new chains on other nodes has lost all the ReFS benefits.

Is there a setting/registry key to make Veeam adhere to much stricter data locality, for use with ReFS?

[Gostev]

No, there's no such setting AFAIK. But what you experienced may be a resource scheduling bug where resource scheduler looks for an extent with enough space for full backup, whereas it should be using previous incremental backups size as a reference whenever next backup is synthetic full to ReFS extent on SOBR with Data Locality. It is worth investigating with support if this is so, or Data Locality policy violation was caused by something else.

[Gostev]

Already confirmed there's no such logic of smart estimation of ReFS synthetic full effective size on disk. But I will give you a workaround shortly.

[ferrus]

That's great, thanks.
Even if the answer was that I had to abandon SOBR, at least I'd know - as the last 48 hours has really broken our ReFS repositories.
I'm manually moving the files back into their respective folders/SOBR nodes - but I don't think moving them uses ReFS links either, so I'm really just chewing up even more space.

[Gostev]

So the workaround is to use SOBRSyntheticFullCompressRate registry value on backup server to control how much actual disk space you expect synthetic fulls to occupy comparing to their file size. The value is in percents, the default (when the registry value is missing) is 100%, meaning v10 will use 100% of the file size of the previous full backup file for extent reservation purposes.

On ReFS/XFS, synthetic fulls can never consume more disk space than sum of all increments (including the one taken on the day when synthetic full is created). So with daily backups, and assuming your daily change rate of 5% (average among our customer base), you can set this value to 5% x 7 days = 35%. This will be the worst case scenario, assuming every incremental backup includes unique blocks - while normally, some blocks will be overwritten every day, which further reduces disk space occupied by the next synthetic full.

[TheOnlyWizard17]

Hi Gostev,

Which Registry-Path to put this registry value exactly ?

[Gostev]

Same path as pretty much any other registry value in Veeam
HKLM\SOFTWARE\Veeam\Veeam Backup and Replication

By the way, I don't know what I was thinking when writing my previous post, or if I was drunk... because what I said does not make any sense. Synthetic fulls on ReFS/XFS do not take ANY physical disk space (except a few hundreds MB for metadata), so you want to set this value to 1% which is the minimum possible.

[tjurgens]

Would this registry key work the same for Cloud Connect repositories?

Also, this post says to stick closer to 50%? I am assuming that's a bit outdated and setting it to 1% is good?

[Gostev]

The registry key works for any SOBR.
See my previous post regarding 50% being an incorrect suggestion

[tjurgens]

Thanks for the clarity Gostev, I meant to link this post in my reply above mentioning the setting being set to 0:
veeam-backup-replication-f2/sorb-backup ... 57990.html

I will go off the info in this thread.

[b.vanhaastrecht]

Hello,

We are currently running low on free diskspace on our 4 extends of a SOBR. We have noticed some behavior that makes us believe SOBR does not calculate with ReFS block cloning. One VM which is 3TB in size has an existing chain on an extend. When this extend has less then 3TB free, SOBR puts the next synthetic full on a other extend. This causes data explosion as the new full will be the start of a new ReFS chain.

Then data explosion happens as all the big fulls of other VM's start to be placed in other extend then their original ones starting a snow ball effect.

Is it true SOBR does not take block cloning into account when calculating for free space?

Kind regards,
Bastiaan

[foggy]

Hi Bastiaan, space estimation is done at the beginning of the backup, when Veeam B&R doesn't know the future savings yet, and it is not able to predict how the data will be deduplicated. However, there's a workaround - see above.

[b.vanhaastrecht]

Hello, than kyo ufor this, I will look into it.

Just a question before I get into it. Why is this an optional key as like Gostev already mentioned, if ReFS the new synthetic full takes les then 1%. Looks like a logic which could be implemented by default.

[foggy]

Well, there's always a place for improvement. The key is global though, not related to any specific file system, so additional logic would be required anyway.

[Gostev]

Looks like a logic which could be implemented by default.

It is implemented as the default in v11.

[b.vanhaastrecht]

So this key becomes irrelevant in V11? Or just for ReFS and Synthetic full?

[Gostev]

Just for ReFS/XFS and synthetic full.