Comprehensive data protection for all workloads
Post Reply
ferrus
Veeam ProPartner
Posts: 268
Liked: 39 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

SOBR Data Locality

Post by ferrus »

Is there a way of enforcing a stricter data locality when using SOBR and ReFS?

We have a three 55TB node SOBR, each with a 6-8TB free space margin when fully loaded.
Because each backup job isn't completely even and data amounts change - we've never been able to keep the free space on each node exactly the same, but it rarely differs by much more than 2-3TB.

Recently, I've been reading more an more on the forums here, that synthetic fulls should really be enabled for ReFS repositories - as they are effectively free and increase efficiency by lowering the backup chain length.
I think I may have encountered this situation before and returned to Forever Incremental, but within 48 hours of enabling synthetic fulls on the SOBR jobs - the data locality rules have been broken, full backups have been introduced on other nodes and the free space has been wiped out because of the duplication of backup files across nodes.

Veeam may not see this as breaking data locality as it hasn't split the backup chain (vbks and vibs) across the nodes, but creating new chains on other nodes has lost all the ReFS benefits.

Is there a setting/registry key to make Veeam adhere to much stricter data locality, for use with ReFS?

Gostev
SVP, Product Management
Posts: 27097
Liked: 4433 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOBR Data Locality

Post by Gostev »

No, there's no such setting AFAIK. But what you experienced may be a resource scheduling bug where resource scheduler looks for an extent with enough space for full backup, whereas it should be using previous incremental backups size as a reference whenever next backup is synthetic full to ReFS extent on SOBR with Data Locality. It is worth investigating with support if this is so, or Data Locality policy violation was caused by something else.

Gostev
SVP, Product Management
Posts: 27097
Liked: 4433 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOBR Data Locality

Post by Gostev »

Already confirmed there's no such logic of smart estimation of ReFS synthetic full effective size on disk. But I will give you a workaround shortly.

ferrus
Veeam ProPartner
Posts: 268
Liked: 39 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

Re: SOBR Data Locality

Post by ferrus »

That's great, thanks.
Even if the answer was that I had to abandon SOBR, at least I'd know - as the last 48 hours has really broken our ReFS repositories.
I'm manually moving the files back into their respective folders/SOBR nodes - but I don't think moving them uses ReFS links either, so I'm really just chewing up even more space.

Gostev
SVP, Product Management
Posts: 27097
Liked: 4433 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOBR Data Locality

Post by Gostev »

So the workaround is to use SOBRSyntheticFullCompressRate registry value on backup server to control how much actual disk space you expect synthetic fulls to occupy comparing to their file size. The value is in percents, the default (when the registry value is missing) is 100%, meaning v10 will use 100% of the file size of the previous full backup file for extent reservation purposes.

On ReFS/XFS, synthetic fulls can never consume more disk space than sum of all increments (including the one taken on the day when synthetic full is created). So with daily backups, and assuming your daily change rate of 5% (average among our customer base), you can set this value to 5% x 7 days = 35%. This will be the worst case scenario, assuming every incremental backup includes unique blocks - while normally, some blocks will be overwritten every day, which further reduces disk space occupied by the next synthetic full.

TheOnlyWizard17
Service Provider
Posts: 29
Liked: 13 times
Joined: Nov 15, 2016 3:38 pm
Full Name: Bart van de Beek
Contact:

Re: SOBR Data Locality

Post by TheOnlyWizard17 »

Hi Gostev,

Which Registry-Path to put this registry value exactly ?

Gostev
SVP, Product Management
Posts: 27097
Liked: 4433 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOBR Data Locality

Post by Gostev » 1 person likes this post

Same path as pretty much any other registry value in Veeam :D
HKLM\SOFTWARE\Veeam\Veeam Backup and Replication

By the way, I don't know what I was thinking when writing my previous post, or if I was drunk... because what I said does not make any sense. Synthetic fulls on ReFS/XFS do not take ANY physical disk space (except a few hundreds MB for metadata), so you will want to set this value to 1% (which is minimum possible).

Post Reply

Who is online

Users browsing this forum: DavidRobertson, Majestic-12 [Bot] and 38 guests