Comprehensive data protection for all workloads
Post Reply
obroni
Service Provider
Posts: 131
Liked: 22 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Fast Clone for Linux FS's

Post by obroni »

Hi,

Marvelling at the enhancements coming in v9.5 surrounding ReFS and then reality strikes and I realise we use Linux repositories. Is there any thoughts about trying to implement similar fast clone functionality with Linux FS's. Although I confess to not understanding how the ReFS enhancements work under the hood. I'm guessing that a lot of it must be possible with BTRFS or ZFS as both are COW in nature and support API's for making a clone of a block?
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Fast Clone for Linux FS's

Post by vClintWyckoff »

Nick-
The ReFS enhancements in 9.5 are quite awesome for certain, its actually my favorite and most impactful feature of 9.5. This integration is made possible by leveraging an API that the ReFS 3.1 file system itself provides, BlockClone API. This API is how Microsoft Hyper-V checkpoints operate in Windows Server 2016.

Check out this public MSDN topic for more information
obroni
Service Provider
Posts: 131
Liked: 22 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Fast Clone for Linux FS's

Post by obroni »

Hi Clint,

Thanks for your response. I understand that the new features in v9.5 only work with ReFS in W2016. I was more referring to if Veeam had any plans to implement similar functionality with something like BTRFS, which has ioctls for "btrfs-extent-same" and "clone-range", which I imagine could be used to provide similar functionality.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Fast Clone for Linux FS's

Post by Gostev »

No short term plans, as BTRFS is too new and unproven to be trusted backups.

Funny you should ask though, because we did research ZFS last month specifically for this purpose. ZFS because it has been extremely popular with the users of Linux-based repositories, especially service providers. However, it appeared that ZFS does not yet provide capabilities similar to BlockClone API of ReFS.

Here's some additional information from the internal forum discussion on the same subject:
tsightler wrote:Here's an LWN article from 2014 that talks a little about the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE which were ioctl functions specific to BTRFS, pretty simple stuff really:

https://lwn.net/Articles/581558/

As of kernel 4.5 these functions were brought into the generic VFS level as FICLONE and FICLONERANGE ioctls:

http://man7.org/linux/man-pages/man2/io ... nge.2.html

However, BTRFS remained the only filesystem with support at that time (although NFS v4 CLONE functions were implemented in nfsd to allow support via network), although the XFS developers were working on implementation, which is what made it make sense to bring the calls to the generic VFS layer. Just a little over a month ago you can see the commit to the 4.9-rc kernel tree for XFS to implement these ioctls:

https://git.kernel.org/cgit/linux/kerne ... d16565c1bd

So perhaps once these have time to settle in, and a few major distros move to these newer kernels, it might be something that could be considered, but at this point I' think it is probably too early for sure.
obroni
Service Provider
Posts: 131
Liked: 22 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Fast Clone for Linux FS's

Post by obroni »

Excellent news, glad you guys are looking at stuff like this. Agree on BTRFS stability, but XFS supporting this as well would be good once it lands. Thanks for the insight and will keep my fingers crossed.
hobbit
Influencer
Posts: 11
Liked: 4 times
Joined: Jan 08, 2013 11:02 am
Contact:

Feature Request: Fast Cloning with Linux filesystems

Post by hobbit »

Hi,

I was told be Veeam support to use this forum for feature requests and just found this thread about the same topic.

We don´t use Windows 2016 backup repositories and so can´t use the current fast clone implementation based on ReFS. It would be great if this feature could be extended to support Linux Filesystems used on Linux Backup Repositories.
Since the last post on this topic was from 11/2016 is there something new, maybe considering XFS Reflink?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Fast Clone for Linux FS's

Post by Gostev »

Last we looked it was still experimental in XFS. Do you know if this changed recently?
hobbit
Influencer
Posts: 11
Liked: 4 times
Joined: Jan 08, 2013 11:02 am
Contact:

Re: Fast Clone for Linux FS's

Post by hobbit » 1 person likes this post

Yes, XFS reflink is no longer tagged "experimental" as of Kernel 4.16.

https://git.kernel.org/torvalds/c/1e369 ... 39700d8884
mweissen13
Enthusiast
Posts: 93
Liked: 54 times
Joined: Dec 28, 2017 3:22 pm
Full Name: Michael Weissenbacher
Contact:

Re: Fast Clone for Linux FS's

Post by mweissen13 »

+1 from me for XFS reflink support
first-hand experience: even in the time when the feature was still marked "experimental" in the Linux kernel, we had ZERO issues with it. While ReFS is still having problems despite being "production-quality" since the release of Windows Server 2016... as seen in https://forums.veeam.com/veeam-backup-r ... 40629.html
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Fast Clone for Linux FS's

Post by tsightler » 1 person likes this post

The experimental tag in the Linux kernel does not mean that a feature is unstable, only that there is not yet enough feedback from real world deployments that the feature is fully stable and thus it's more of a warning that you are an early adopter. It's no different than new features from Microsoft, except they don't actually give you the warning in most cases. In both cases, you are an early adopter, and so more bugs than normal are likely to be expected. The good thing about being an early adopter of features in the Linux kernel, you can post straight to the devs if you have issues and they'll probably take an immediate interest and usually provide patches in a matter of hours to days.
davow
Influencer
Posts: 22
Liked: 2 times
Joined: Jan 25, 2013 5:05 pm
Full Name: D W
Contact:

[MERGED] RHEL 8 and Copy on Write XFS (Fast Clone)

Post by davow »

Hi,
Now that RHEL 8 has been released (with CentOS probably 8 weeks away), do Veeam have plans to implement the new XFS features? (shared copy-on-write data extents are supported by XFS)
Unless I'm mistaken, this would allow Veeam to provide similar space reductions that ReFS currently enjoys in Win2016/19. ie fast Clone.
I recall a forum thread from Anton a couple of years ago predicting this but wanted to know if this was on the Veeam roadmap.

Here is the thread...veeam-backup-replication-f2/fast-clone- ... 38841.html
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Fast Clone for Linux FS's

Post by HannesK »

Hello,
I merged your post with the old thread.

yes, we have some plans for that but I cannot promise any timeline.

Best regards,
Hannes
davow
Influencer
Posts: 22
Liked: 2 times
Joined: Jan 25, 2013 5:05 pm
Full Name: D W
Contact:

Re: Fast Clone for Linux FS's

Post by davow »

Ok, thanks.
Post Reply

Who is online

Users browsing this forum: Brian.Knoblauch, Google [Bot], Semrush [Bot] and 229 guests