Comprehensive data protection for all workloads
Post Reply
jb81
Novice
Posts: 6
Liked: never
Joined: Oct 24, 2014 4:01 pm
Full Name: JB81
Contact:

Understanding Integration with ReFS and FastCloning

Post by jb81 »

Hi all, I made a simple lab to test the integration of veeam with ReFS: a backup server with Windows Server 2016 Standard, a Data Volume formatted in ReFS, and Veeam Backup&Replication 9.5update1.
I created 2 repositories on the ReFS Volume.
I configured a simple backup job (1vm) targeted the the first repository, and it runs fine (first full is about 15GB) and it took around 15 minutes.
Then I configured a backup copy job targeted to the second repository, and selected the backup job as source. It runs fine but I was expecting this job to be very quick (due to the fast clone APIs). Insted it took 15 minutes, copying all of the blocks of the original file, as it would be on a regular NTFS volume....
Is there something wrong or didn't I understand how it works with Veeam and ReFS integration?

Many thanks in advance.
Vandelete
Lurker
Posts: 1
Liked: never
Joined: Dec 27, 2015 6:58 pm
Full Name: Wiwit van Wissen
Location: The Netherlands
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by Vandelete »

Hi jb81,

Read the next article Written by Luca for a more in depth look for ReFS integration.
Maybe this was the misinterpreted expected result.
I guess you where looking after "a-few-seconds-later" backup-copy-job succesfull...
but it does not quite function that way you wanted or expected it too, also in regards a "clone" as where you expected ReFS clone-functionality in a "back-up copy job".
Not to be mistaken with thin clone functionality at the storage level in terms of the Back-up copy job wich is a 1:1 relation to the original backup file as well in copied size.

http://www.virtualtothecore.com/en/wind ... repository
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by vClintWyckoff » 2 people like this post

First recommendation is to use 64K cluster size when you originally formatted the ReFS volume, not the default of 4K.

Second, Veeam Backup & Replication leverages Fast Clone for the following synthetic operations:

In backup jobs
•Merge of backup files
•Synthetic full backup
•Reverse incremental backup transformation
•Compact of full backup file

In backup copy job
•Merge of backup files
•Creation of archive full backups (synthetic method)
•Compact of full backup file

So in your case below on the backup job itself you'd see [fast clone] improvements when a Synthetic operation occurs or if you chose Reverse Incremental as the backup type when you created the job. Or on the BCJ you'd see [fast clone] improvements in the merge or in the GFS retention creation (4 Weekly Full, 12 Monthly Full...etc) or in compact.

Hope this helps.
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by lowlander »

Thanks for explaining. Is it correct if we state that you will save storage when using a GFS schedule ? For example, a monthly backup is a pointer to a full backup with al its increments ?
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by vClintWyckoff » 1 person likes this post

BCJ backups that benefit from the ReFS Space Savings are on the GFS portion. So to explain this - in the UI there's the check boxes that allow you to say I want to keep 4 Weekly Fulls, 12 Monthly Fulls etc. That is where you will see [fast clone].
EricB
Influencer
Posts: 12
Liked: 1 time
Joined: Jul 13, 2016 6:55 am
Full Name: Eric
Location: The Netherlands
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by EricB »

Hi guys, another question to further understand the way Veeam integrates with ReFS.
1. Is it required that both the repository for the primary backup and the backup copy are stored on ReFS?
2. If yes, would they (both repositories) be required to be located on the same Windows ReFS volume?

I got confused about this due to the following "Fast Clone requires that source and destination files are stored on the same ReFS volume." located on: https://helpcenter.veeam.com/docs/backu ... tml?ver=95

Reason for asking is because my customer are typically slow adopters that would ideally have a primary backup on NTFS and only the backup copy on ReFS (traditionally the big storage consumer with GFS schedules and long retention policies).

Thanks
tdewin
Veeam Software
Posts: 1775
Liked: 646 times
Joined: Mar 02, 2012 1:40 pm
Full Name: Timothy Dewin
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by tdewin »

Block clone api (the MS underlying API) only works in the same filesystem. This is something that you need to keep in mind when you are using scale out because your files can be scattered over multiple filesystems (or volumes)

I'm pretty sure that we also, only do the fast clone inside the same backup chain. But anyway, when you use default GFS (and not "reread data from source"), the GFS points are constructed based on the existing backup copy chain (well technically the new full for the existing chain is built on this data, the GFS point is just put aside but the point is that they are all linked together)
EricB
Influencer
Posts: 12
Liked: 1 time
Joined: Jul 13, 2016 6:55 am
Full Name: Eric
Location: The Netherlands
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by EricB »

So if i understand this correctly, the Backup Copy job with GFS would not use blockcloning technology if the primary backup job data is located on NTFS?
Then it's really a decision whether you go full on NTFS for both primary and secundary repositories or full on REFS for both.
tdewin
Veeam Software
Posts: 1775
Liked: 646 times
Joined: Mar 02, 2012 1:40 pm
Full Name: Timothy Dewin
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by tdewin »

GFS will still use block clone but only to link to blocks in it's own proper chain. It will not link to block of the source job. So it doesn't matter if the primary job is on NTFS, ReFS or on the same ReFS volume.
DaveWatkins
Veteran
Posts: 370
Liked: 97 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by DaveWatkins »

EricB wrote:So if i understand this correctly, the Backup Copy job with GFS would not use blockcloning technology if the primary backup job data is located on NTFS?
Then it's really a decision whether you go full on NTFS for both primary and secundary repositories or full on REFS for both.
Backup jobs, and Backup Copy jobs are not linked at all via ReFS. The purpose of a copy job is that it's a complete copy and not dependent on the backup job in any way, that way if your backup job becomes corrupt the copy job still has good data. Even if they are on the same volume there is no ReFS fast clone integration between them. The ReFS integration happens within a job/backup chain only
ITP-Stan
Service Provider
Posts: 201
Liked: 55 times
Joined: Feb 18, 2013 10:45 am
Full Name: Stan (IF-IT4U)
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by ITP-Stan »

The ReFS block clone benefits do not spread out over multiple jobs.
"Fast Clone requires that source and destination files are stored on the same ReFS volume."
This does not refer to source backup job and destination backup copy job. This means for blockcloning to work all files (all incrementals and fulls for 1 chain or 1 job) need to be on the same volume, with scale-out repository it can be spread across multiple volumes.
sandsturm
Veteran
Posts: 278
Liked: 23 times
Joined: Mar 23, 2015 8:30 am
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by sandsturm »

I have a backup repository on ReFS and another one for the Backup copy Job also on ReFS. A VM is backed up to this backup repository and a backup copy job is configured to make a copy of this VM once a month. It's clear that the blockcloning works only within a Repo. My questions is the following, if I have multiple monthly copies in the backup copy repository, is the blockcloning in this case in use for all these monthly copies?

thx
sandsturm
ITP-Stan
Service Provider
Posts: 201
Liked: 55 times
Joined: Feb 18, 2013 10:45 am
Full Name: Stan (IF-IT4U)
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by ITP-Stan »

I use GFS on my BackupCopyJob wich keeps 3 quarterly, 3 monthly and 3 weekly virtual fulls.
They all use ReFS block cloning which saves a huge amount of disk space.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Understanding Integration with ReFS and FastCloning

Post by foggy »

Correct, Fast Clone works for GFS backups creation in case they are created synthetically.
Post Reply

Who is online

Users browsing this forum: No registered users and 169 guests