Fast Clone for Linux FS's

Availability for the Always-On Enterprise

Fast Clone for Linux FS's

Veeam Logoby obroni » Tue Nov 15, 2016 4:14 pm

Hi,

Marvelling at the enhancements coming in v9.5 surrounding ReFS and then reality strikes and I realise we use Linux repositories. Is there any thoughts about trying to implement similar fast clone functionality with Linux FS's. Although I confess to not understanding how the ReFS enhancements work under the hood. I'm guessing that a lot of it must be possible with BTRFS or ZFS as both are COW in nature and support API's for making a clone of a block?
obroni
Service Provider
 
Posts: 102
Liked: 10 times
Joined: Fri Nov 21, 2014 10:50 pm
Full Name: Nick Fisk

Re: Fast Clone for Linux FS's

Veeam Logoby vClintWyckoff » Tue Nov 15, 2016 5:09 pm

Nick-
The ReFS enhancements in 9.5 are quite awesome for certain, its actually my favorite and most impactful feature of 9.5. This integration is made possible by leveraging an API that the ReFS 3.1 file system itself provides, BlockClone API. This API is how Microsoft Hyper-V checkpoints operate in Windows Server 2016.

Check out this public MSDN topic for more information
vClintWyckoff
Expert
 
Posts: 500
Liked: 109 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: Fast Clone for Linux FS's

Veeam Logoby obroni » Tue Nov 15, 2016 5:15 pm

Hi Clint,

Thanks for your response. I understand that the new features in v9.5 only work with ReFS in W2016. I was more referring to if Veeam had any plans to implement similar functionality with something like BTRFS, which has ioctls for "btrfs-extent-same" and "clone-range", which I imagine could be used to provide similar functionality.
obroni
Service Provider
 
Posts: 102
Liked: 10 times
Joined: Fri Nov 21, 2014 10:50 pm
Full Name: Nick Fisk

Re: Fast Clone for Linux FS's

Veeam Logoby Gostev » Tue Nov 15, 2016 10:45 pm

No short term plans, as BTRFS is too new and unproven to be trusted backups.

Funny you should ask though, because we did research ZFS last month specifically for this purpose. ZFS because it has been extremely popular with the users of Linux-based repositories, especially service providers. However, it appeared that ZFS does not yet provide capabilities similar to BlockClone API of ReFS.

Here's some additional information from the internal forum discussion on the same subject:
tsightler wrote:Here's an LWN article from 2014 that talks a little about the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE which were ioctl functions specific to BTRFS, pretty simple stuff really:

https://lwn.net/Articles/581558/

As of kernel 4.5 these functions were brought into the generic VFS level as FICLONE and FICLONERANGE ioctls:

http://man7.org/linux/man-pages/man2/io ... nge.2.html

However, BTRFS remained the only filesystem with support at that time (although NFS v4 CLONE functions were implemented in nfsd to allow support via network), although the XFS developers were working on implementation, which is what made it make sense to bring the calls to the generic VFS layer. Just a little over a month ago you can see the commit to the 4.9-rc kernel tree for XFS to implement these ioctls:

https://git.kernel.org/cgit/linux/kerne ... d16565c1bd

So perhaps once these have time to settle in, and a few major distros move to these newer kernels, it might be something that could be considered, but at this point I' think it is probably too early for sure.
Gostev
Veeam Software
 
Posts: 22379
Liked: 2670 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Fast Clone for Linux FS's

Veeam Logoby obroni » Wed Nov 16, 2016 2:41 pm

Excellent news, glad you guys are looking at stuff like this. Agree on BTRFS stability, but XFS supporting this as well would be good once it lands. Thanks for the insight and will keep my fingers crossed.
obroni
Service Provider
 
Posts: 102
Liked: 10 times
Joined: Fri Nov 21, 2014 10:50 pm
Full Name: Nick Fisk

Feature Request: Fast Cloning with Linux filesystems

Veeam Logoby hobbit » Fri Jun 01, 2018 12:03 pm

Hi,

I was told be Veeam support to use this forum for feature requests and just found this thread about the same topic.

We don´t use Windows 2016 backup repositories and so can´t use the current fast clone implementation based on ReFS. It would be great if this feature could be extended to support Linux Filesystems used on Linux Backup Repositories.
Since the last post on this topic was from 11/2016 is there something new, maybe considering XFS Reflink?
hobbit
Novice
 
Posts: 8
Liked: 1 time
Joined: Tue Jan 08, 2013 11:02 am

Re: Fast Clone for Linux FS's

Veeam Logoby Gostev » Fri Jun 01, 2018 3:51 pm

Last we looked it was still experimental in XFS. Do you know if this changed recently?
Gostev
Veeam Software
 
Posts: 22379
Liked: 2670 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Fast Clone for Linux FS's

Veeam Logoby hobbit » Mon Jun 04, 2018 7:35 am 1 person likes this post

Yes, XFS reflink is no longer tagged "experimental" as of Kernel 4.16.

https://git.kernel.org/torvalds/c/1e369 ... 39700d8884
hobbit
Novice
 
Posts: 8
Liked: 1 time
Joined: Tue Jan 08, 2013 11:02 am

Re: Fast Clone for Linux FS's

Veeam Logoby mweissen13 » Mon Jun 04, 2018 8:17 am

+1 from me for XFS reflink support
first-hand experience: even in the time when the feature was still marked "experimental" in the Linux kernel, we had ZERO issues with it. While ReFS is still having problems despite being "production-quality" since the release of Windows Server 2016... as seen in veeam-backup-replication-f2/refs-4k-horror-story-t40629.html
mweissen13
Service Provider
 
Posts: 15
Liked: 6 times
Joined: Thu Dec 28, 2017 3:22 pm
Full Name: Michael Weissenbacher

Re: Fast Clone for Linux FS's

Veeam Logoby tsightler » Mon Jun 04, 2018 1:16 pm

The experimental tag in the Linux kernel does not mean that a feature is unstable, only that there is not yet enough feedback from real world deployments that the feature is fully stable and thus it's more of a warning that you are an early adopter. It's no different than new features from Microsoft, except they don't actually give you the warning in most cases. In both cases, you are an early adopter, and so more bugs than normal are likely to be expected. The good thing about being an early adopter of features in the Linux kernel, you can post straight to the devs if you have issues and they'll probably take an immediate interest and usually provide patches in a matter of hours to days.
tsightler
Veeam Software
 
Posts: 5090
Liked: 2010 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Gostev, mschwaermer, poulpreben, stevenvangeel and 60 guests