Fast Clone for Linux FS's

[obroni]

Hi,

Marvelling at the enhancements coming in v9.5 surrounding ReFS and then reality strikes and I realise we use Linux repositories. Is there any thoughts about trying to implement similar fast clone functionality with Linux FS's. Although I confess to not understanding how the ReFS enhancements work under the hood. I'm guessing that a lot of it must be possible with BTRFS or ZFS as both are COW in nature and support API's for making a clone of a block?

[vClintWyckoff]

Nick-
The ReFS enhancements in 9.5 are quite awesome for certain, its actually my favorite and most impactful feature of 9.5. This integration is made possible by leveraging an API that the ReFS 3.1 file system itself provides, BlockClone API. This API is how Microsoft Hyper-V checkpoints operate in Windows Server 2016.

Check out this public MSDN topic for more information

[obroni]

Hi Clint,

Thanks for your response. I understand that the new features in v9.5 only work with ReFS in W2016. I was more referring to if Veeam had any plans to implement similar functionality with something like BTRFS, which has ioctls for "btrfs-extent-same" and "clone-range", which I imagine could be used to provide similar functionality.

[Gostev]

No short term plans, as BTRFS is too new and unproven to be trusted backups.

Funny you should ask though, because we did research ZFS last month specifically for this purpose. ZFS because it has been extremely popular with the users of Linux-based repositories, especially service providers. However, it appeared that ZFS does not yet provide capabilities similar to BlockClone API of ReFS.

Here's some additional information from the internal forum discussion on the same subject:

Here's an LWN article from 2014 that talks a little about the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE which were ioctl functions specific to BTRFS, pretty simple stuff really:

https://lwn.net/Articles/581558/

As of kernel 4.5 these functions were brought into the generic VFS level as FICLONE and FICLONERANGE ioctls:

http://man7.org/linux/man-pages/man2/io ... nge.2.html

However, BTRFS remained the only filesystem with support at that time (although NFS v4 CLONE functions were implemented in nfsd to allow support via network), although the XFS developers were working on implementation, which is what made it make sense to bring the calls to the generic VFS layer. Just a little over a month ago you can see the commit to the 4.9-rc kernel tree for XFS to implement these ioctls:

https://git.kernel.org/cgit/linux/kerne ... d16565c1bd

So perhaps once these have time to settle in, and a few major distros move to these newer kernels, it might be something that could be considered, but at this point I' think it is probably too early for sure.

[obroni]

Excellent news, glad you guys are looking at stuff like this. Agree on BTRFS stability, but XFS supporting this as well would be good once it lands. Thanks for the insight and will keep my fingers crossed.

[hobbit]

Hi,

I was told be Veeam support to use this forum for feature requests and just found this thread about the same topic.

We don´t use Windows 2016 backup repositories and so can´t use the current fast clone implementation based on ReFS. It would be great if this feature could be extended to support Linux Filesystems used on Linux Backup Repositories.
Since the last post on this topic was from 11/2016 is there something new, maybe considering XFS Reflink?

[Gostev]

Last we looked it was still experimental in XFS. Do you know if this changed recently?

[hobbit]

Yes, XFS reflink is no longer tagged "experimental" as of Kernel 4.16.

https://git.kernel.org/torvalds/c/1e369 ... 39700d8884

[mweissen13]

+1 from me for XFS reflink support
first-hand experience: even in the time when the feature was still marked "experimental" in the Linux kernel, we had ZERO issues with it. While ReFS is still having problems despite being "production-quality" since the release of Windows Server 2016... as seen in https://forums.veeam.com/veeam-backup-r ... 40629.html

[tsightler]

The experimental tag in the Linux kernel does not mean that a feature is unstable, only that there is not yet enough feedback from real world deployments that the feature is fully stable and thus it's more of a warning that you are an early adopter. It's no different than new features from Microsoft, except they don't actually give you the warning in most cases. In both cases, you are an early adopter, and so more bugs than normal are likely to be expected. The good thing about being an early adopter of features in the Linux kernel, you can post straight to the devs if you have issues and they'll probably take an immediate interest and usually provide patches in a matter of hours to days.

[davow]

Hi,
Now that RHEL 8 has been released (with CentOS probably 8 weeks away), do Veeam have plans to implement the new XFS features? (shared copy-on-write data extents are supported by XFS)
Unless I'm mistaken, this would allow Veeam to provide similar space reductions that ReFS currently enjoys in Win2016/19. ie fast Clone.
I recall a forum thread from Anton a couple of years ago predicting this but wanted to know if this was on the Veeam roadmap.

Here is the thread...veeam-backup-replication-f2/fast-clone- ... 38841.html

[HannesK]

Hello,
I merged your post with the old thread.

yes, we have some plans for that but I cannot promise any timeline.

Best regards,
Hannes

[davow]

Ok, thanks.