Feature Request: REFS Synthetic Transform logic change

Availability for the Always-On Enterprise

Feature Request: REFS Synthetic Transform logic change

Veeam Logoby billcouper » Tue Jun 12, 2018 1:54 am 1 person likes this post


REFS is a great feature. I don't think I could live without it. It is a huge pain in the nut though.
Of all the 'problems' or 'missing features' that REFS has, the only one that is truly in Veeams control is:

1) VBR doesn't seem to understand block-cloning when performing synthetic operations
When creating Synthetic Full backups using block-cloning, virtually no disk space is consumed - only metadata pointers to existing blocks are created. However Veeam refuses to create synthetic points if there isn't "enough" disk space available. Even after tweaking the SOBRTransformRate registry value, it still doesn't realize there is actually enough space to complete the operation.
If the target repository is REFS and a synthetic operation is taking place, Veeam shouldn't use the standard rules for determining if there is enough space.

The other issues I have with REFS are more likely to fall with Microsoft, but perhaps Veeam could mention it to Microsoft devs?

2) How much REAL ACTUAL disk space does a block-cloned backup chain comprising multiple full backups consume? This is an absolute nightmare for us as a company when it comes to billing customers. Synthetic full restore points blow out a customers "disk usage" even though they don't really contain any additional data over an increment. There have been multiple suggestions and community efforts such as 'blockstat' however I've not been able to find anything that is even remotely reliable at calculating actual disk usage. If anyone has any suggestions I am all ears! This is actually our single largest issue with REFS and if we had known before-hand may have had an impact on our decision to use it.

3) Moving block-cloned files intact without rehydration. It would be very handy to be able to move block-cloned backup chains from one REFS volume to another REFS volume without them being rehydrated during the process.
Posts: 18
Liked: 6 times
Joined: Mon Dec 18, 2017 8:58 am
Full Name: Bill Couper

Re: Feature Request: REFS Synthetic Transform logic change

Veeam Logoby Gostev » Tue Jun 12, 2018 2:53 pm

Hi, Bill

1. This would be a bug, as I believe we do have a special logic for dedupe storage - so sounds like it does not apply to ReFS.

2. In theory I think we could track copied vs. referenced blocks when doing synthetic fulls, and store real consumed space with the VBK. The problem however is that when oldest VBK (owner of most referenced blocks) goes away by retention, the usage report will become incorrect. So indeed this is a challenging issue to solve... let's brainstorm this together.

3. This is already in our short-term roadmap because this capability is needed for so many things.

Veeam Software
Posts: 22187
Liked: 2613 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Feature Request: REFS Synthetic Transform logic change

Veeam Logoby dellock6 » Tue Jun 12, 2018 3:19 pm

I think one starting point to discuss the sizing calculations for ReFS could come exactly from the blockstat tool that Timo wrote, and the value it shows. The main split I see is between blocks that are stored only once, so they actually belong to the file, and anything that belongs to at least two references, which means we are saving space. So an information about the file could be about these two parameters, exclusive (consumed) space + shares (consumed) space. Probably the sum of all the exclusive space of a chain is the value you want to bill a customer for, plus the entire value of a full backup file which is the starting point and probably has most of the shared space.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

vExpert 2011-2012-2013-2014-2015-2016-2017-2018
Veeam VMCE #1
Veeam Software
Posts: 5353
Liked: 1454 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Return to Veeam Backup & Replication

Who is online

Users browsing this forum: Bing [Bot], gummett and 54 guests