REFS CBT

[petergrande]

Hi,

I started using veeam agent on a physical machine with about 5 TB of data on a REFS volume, when doing an whole computer backup all drives that are NTFS scans fast with the helpof CBT, but when it comes to the REFS volume it takes about 7-8 hours/day to scan, i suspect that theres no CBT for REFS yet, also if I understand things correctly RCT is only available for VMs and not on any REFS volume?

Question: is this normal behaviour with REFS volumes? or am I´m doing something wrong?

regards
/Peter

[Dima P.]

Hi Peter,

What was the resulting backup side of the REFS volume (full and incremental)? Thanks.

[jabrams2017]

I am also having this issue. No error messages that I can see in the logs and the backups are successful. Physical machine, Windows 10 1702 x64. The main OS volume is NTFS and does CBT just fine. My data volume is REFS v1.2, 2-way mirror, 6TB thin provisioned with plenty of space left in the storage pool. I created a new mirrored storage space, same pool, but with REFS v3.2 and encountered the same issue. I'm guessing the OP is correct in that CBT isn't supported on REFS volumes, but I can't find that anywhere in the documentation I've found. Any help with this would be greatly appreciated. Thanks!

[Dima P.]

Hi guys,

Kindly submit a support cases via Control Panel > Support > Technical Support. I check with QA lately and they confirmed that CBT should work on the REFS volumes specified as a target for backup. Please do not forget to share the case IDs in this thread. Thank you.

[jabrams2017]

My case ID is 02271451. I will follow up when I get something from the support team. This is a great product and if it works for REFS volumes it would be perfect for my needs.

[Dima P.]

Cool! Thanks for the heads up. Let us know how it goes.

[jabrams2017]

The support team has hit a bit of a brick wall. They are trying to recreate the situation in the lab and will let me know their findings. I have tried a few different test ReFS volumes using fixed and thin provisioning and have not been able to get Veeam Agent to use CBT on any of these volumes. Since I am only backing up around 3TB of data, the backup without CBT is around 6 hours. This will work for now until they figure out what the problem is, but if I add more data the read times could get a little out of hand. If anyone has any suggestions, I am willing to do some more testing myself.

[Dima P.]

Thanks for the update. I checked with the team and here are the results - in case of the ReFS volume we cannot leverage MTF change block tracking, so VAW should re-read used space of the entire volume to build the proper increment. This is the reason why incremental backup takes, so much time in case of ReFS partition.

[jabrams2017]

I have uploaded my logs to the support team to continue troubleshooting. Will update when they have more information. I have done a test restore of the ReFS volume from the VAW backups and verified they are working correctly, just not leveraging CBT.

[jabrams2017]

According to the support team, it seems that CBT will not work with ReFS volumes on Windows 10. The whole volume must be read for each incremental. They said that because of the slow IO on one of the drives in the Storage Pool the ReFS can't use CBT. I've asked some questions of them and I'm waiting to hear back. I would be interested if anyone has successfully used CBT for any ReFS volume, be it directly on a single physical disk or storage pool. I'll post their statement along with my questions below. I'll also post their response to these questions when I receive it.

Suport Team statement:
"We have heard back from research and testing. They have found that while Windows 10 storage spaces will work, the average processing read/write speed was lower than that of the read/write speed of the slowest disk during tests. The extra time it takes to read 3TB appears to be expected behavior for a Windows 10 storage spaces setup.

Let me know if you have any further questions."

My follow up questions:
"I guess I'm confused and could use some clarification. VAW is supposed to support CBT on ReFS volumes, but because of the I/O deficiency of the physical disks in the storage pool it won't work in Windows 10? How does VAW even interact with the physical disks directly when it is presented with a virtual disk by Windows? Would CBT work on the ReFS volume if I was running Windows Server 2016 instead? And why does CBT work on a NTFS volume in the same storage pool if the pool itself is the problem? Any clarification would be greatly appreciated."

Just to be clear, I think the Veeam Support Team has done a great job troubleshooting this issue so far. They are professional and quick to respond. If it turns out that ReFS volumes just don't support CBT, then maybe that is something that can be addressed in the next release. I'm sure many users aren't running production ReFS volumes, but it will be more common as time goes on considering the features the file system offers.

[Technogod]

I know this is off topic but could be an issue when trying to restore. Have you tried to do a recovery booting from recovery media. When I tried it the system was unable to see the ReFS hard drive. I am also running Windows 10 ver. 1703.

[jabrams2017]

@Technogod
I haven't played around with the recovery media as of yet (at least not any more than creating a bootable DVD). Only started playing around with VAW when looking for a more robust backup solution than the robocopy scripts I was running on my main data volume. I have done test restores on the file level and volume level within the OS and they were fine. Was really hoping the CBT would work on the large ReFS volume to cut back on incremental backup times.

[jabrams2017]

As it turns out, VAW CBT DOES NOT work with ReFS volumes of any kind due to the file system lacking a MFT which is required for this feature to work. This is disappointing but understandable. I just wish this had been told to me up front or just after opening the ticket to save everyone's time. Please consider adding this to documentation since the CBT function does work perfectly with NTFS volumes.

[Dima P.]

jabrams2017,

Thank you. We will add this information to the Help Center and User Guide for Veeam Agent for Windows.

[MMDP]

In Server 2016 is same problem. Imagine everyday process 18TB of data.
It is 7h everyday just read one ReFS volume. It is BIG problem.

[Dima P.]

Hi Michal,

We are currently working on solution for backups made on systems running Windows Servers.

[PC535]

I realize this is an older thread but am wondering if any progress had been made. I noticed my 5TB of backups on a ReFS volume (Server 2016) do take about 9 hours even if there are only a few GB of changes. I am using Veeam Agent for Windows 2.1.0.423 - Free and was wondering in the paid version have this functionality

[Dima P.]

Hi Peter.

Yes it possible with Veeam Agent for Windows 2.1 operating in Server edition: you can install the newly added CBT Driver to avoid full reads of your ReFS volume.