Comprehensive data protection for all workloads
Post Reply
billcouper
Service Provider
Posts: 150
Liked: 30 times
Joined: Dec 18, 2017 8:58 am
Full Name: Bill Couper
Contact:

Feature Request: REFS Synthetic Transform logic change

Post by billcouper » 1 person likes this post

Hi,

REFS is a great feature. I don't think I could live without it. It is a huge pain in the nut though.
Of all the 'problems' or 'missing features' that REFS has, the only one that is truly in Veeams control is:

1) VBR doesn't seem to understand block-cloning when performing synthetic operations
When creating Synthetic Full backups using block-cloning, virtually no disk space is consumed - only metadata pointers to existing blocks are created. However Veeam refuses to create synthetic points if there isn't "enough" disk space available. Even after tweaking the SOBRTransformRate registry value, it still doesn't realize there is actually enough space to complete the operation.
If the target repository is REFS and a synthetic operation is taking place, Veeam shouldn't use the standard rules for determining if there is enough space.




The other issues I have with REFS are more likely to fall with Microsoft, but perhaps Veeam could mention it to Microsoft devs?

2) How much REAL ACTUAL disk space does a block-cloned backup chain comprising multiple full backups consume? This is an absolute nightmare for us as a company when it comes to billing customers. Synthetic full restore points blow out a customers "disk usage" even though they don't really contain any additional data over an increment. There have been multiple suggestions and community efforts such as 'blockstat' however I've not been able to find anything that is even remotely reliable at calculating actual disk usage. If anyone has any suggestions I am all ears! This is actually our single largest issue with REFS and if we had known before-hand may have had an impact on our decision to use it.

3) Moving block-cloned files intact without rehydration. It would be very handy to be able to move block-cloned backup chains from one REFS volume to another REFS volume without them being rehydrated during the process.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature Request: REFS Synthetic Transform logic change

Post by Gostev »

Hi, Bill

1. This would be a bug, as I believe we do have a special logic for dedupe storage - so sounds like it does not apply to ReFS.

2. In theory I think we could track copied vs. referenced blocks when doing synthetic fulls, and store real consumed space with the VBK. The problem however is that when oldest VBK (owner of most referenced blocks) goes away by retention, the usage report will become incorrect. So indeed this is a challenging issue to solve... let's brainstorm this together.

3. This is already in our short-term roadmap because this capability is needed for so many things.

Thansk!
dellock6
VeeaMVP
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Feature Request: REFS Synthetic Transform logic change

Post by dellock6 »

I think one starting point to discuss the sizing calculations for ReFS could come exactly from the blockstat tool that Timo wrote, and the value it shows. The main split I see is between blocks that are stored only once, so they actually belong to the file, and anything that belongs to at least two references, which means we are saving space. So an information about the file could be about these two parameters, exclusive (consumed) space + shares (consumed) space. Probably the sum of all the exclusive space of a chain is the value you want to bill a customer for, plus the entire value of a full backup file which is the starting point and probably has most of the shared space.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
billcouper
Service Provider
Posts: 150
Liked: 30 times
Joined: Dec 18, 2017 8:58 am
Full Name: Bill Couper
Contact:

Re: Feature Request: REFS Synthetic Transform logic change

Post by billcouper »

Thank you both for the reply.

Gostev, you have made me so happy to know that being able to move backup chains between REFS volumes without rehydration is already on the road-map. With the ReFS synthetic operation 'free space' error being a potential bug, should I log a support ticket for that?

dellock6, I found the blockstat utility very very quickly after running the first customer billing report on the new backup system (Veeam) using our existing reporting module - the numbers were so much higher than before I thought I was going to be rich... no, I didn't think that actually, rather, I was stunned like a Deer in headlights. The ability to see 'actual size' of files on disk wasn't even on my radar as a potential shortcoming. When we tested Veeam using REFS we did that internally, on our own servers and we don't bill ourselves so I never ran the billing report generator against the files on disk.... oh boy that was a huge mistake... So I did some google searching and found blockstat. Hallelujah, right? I had a go at using it myself to generate size on disk figures and what I got out of it didn't add up. So I handed the blockstat utility to our lead programmer to dissect and make work for me. Started by doing blockstat against all files per-VM per-extent. Then tried querying Veeam api what files it had fast-cloned together and run blockstat against those files. No matter which way we tried to calculate the size on disk it was always inaccurate. For a single VM the numbers look reasonable. When we calculate the size on disk for all VM's and add them together, they don't match the actual disk consumption (as reported in Disk Management). Some days, on some extents, the numbers were "close enough" but other days, or on other extents, the numbers varied wildly. We ended up giving up on this and hoping that one day Microsoft would eventually build this capability into Explorer (cmon MS, block-clone has been around almost 18 months now!).

If anyone has been able to get accurate figures using blockstat I would really appreciate if you shared your method or code!
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature Request: REFS Synthetic Transform logic change

Post by Gostev »

Yes, let's process the "free space" error as a bug - need to start from support case, then if the issue is confirmed by support they will have devs create an internal bug record. Thanks!
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 259 guests