Comprehensive data protection for all workloads
JoshuaPostSAMC
Expert
Posts: 124
Liked: 22 times
Joined: Jul 30, 2015 7:32 pm
Contact:

Re: VBR 9.5 - REFS

Post by JoshuaPostSAMC »

PTide, would I see increase performance iSCSI vs CIFS in general, or only when paired with ReFS?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev »

Only when paired with ReFS.
chattingh
Enthusiast
Posts: 57
Liked: 3 times
Joined: Jan 22, 2016 12:45 am
Full Name: Carl Hattingh
Contact:

Re: VBR 9.5 - REFS

Post by chattingh »

I have a question about how to seed a backup copy job with an ReFS repository. If I understand correctly from this thread, the Fast Clone advantages of an ReFS repo can only be leveraged once the new VBK files are properly block aligned with the filesystem. To achieve that, an active or synthetic full must be run against the ReFS repository. Is my understanding from reading this thread correct?

That implies to me that simply copying existing backup files to an ReFS repo will not activate the Fast Clone API. How then, would one seed a backup copy job to an ReFS repo (for Cloud Connect as an example) whilst obtaining the benefits of the Fast Clone API?

Apologies if this is in the doco, but I coudn't find it.
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: VBR 9.5 - REFS

Post by vClintWyckoff »

Carl-
Correct, you would need to kick off an active full or a synthetic full to align with the file system to begin to see the benefits. Alternatively, you could do a Backup Copy job with a 1 to 1 match of the original job and after that completes then remap to the newly created backup files.
chattingh
Enthusiast
Posts: 57
Liked: 3 times
Joined: Jan 22, 2016 12:45 am
Full Name: Carl Hattingh
Contact:

Re: VBR 9.5 - REFS

Post by chattingh »

Thanks Clint

To confirm, in the scenario where a Backup Copy job is going over the WAN or Internet, and there is insufficient bandwidth to perform an active full, but instead a data seed is required, are we saying that it is not possible to leverage the Fast Clone API?

We typically do the data seed for clients by targeting the Backup Copy job to removable media or NAS, take it to the destination, copy the backup files to the proper repo, then update the job with the proper repo and map the backpup files.

Is it possible to perform a synthetic full backup on a backup copy job? I don't think so as there are no options in the Backup Copy job wizard.
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: VBR 9.5 - REFS

Post by vClintWyckoff »

To confirm, in the scenario where a Backup Copy job is going over the WAN or Internet, and there is insufficient bandwidth to perform an active full, but instead a data seed is required, are we saying that it is not possible to leverage the Fast Clone API?

We typically do the data seed for clients by targeting the Backup Copy job to removable media or NAS, take it to the destination, copy the backup files to the proper repo, then update the job with the proper repo and map the backpup files.
In this case the customer wouldn't benefit from any [fast merge] operations but any additional jobs that would be run locally against that ReFS volume would benefit. Depending on the external volume, what I have found is that some permit the formatting with ReFS, some do not - but when you do the backup copy to the external media and create the GFS retention, that is a synthetic operation so in that respect BCJ is eligible to benefit from ReFS, but not when you copy off those backup files and they'll literally be copying the real blocks of data.
andyg
Enthusiast
Posts: 58
Liked: 5 times
Joined: Apr 23, 2014 9:51 am
Full Name: Andy Goldschmidt
Contact:

[MERGED] VBR 9.5 - REFS (win2016) - Tips and tricks and gotc

Post by andyg »

Hi Guys

Can anyone using ReFS on Win2016 please paste any war stories, tips and tricks in this thread.

I am about to move to ReFS and any tips, tricks or gotchas would be really good. Any step-by-steps guides would be awesome too !!

Try keep it on topic please. :D
-= VMCE v9 certified =-
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: VBR 9.5 - REFS (win2016) - Tips and tricks and gotchas

Post by vClintWyckoff »

Andy: There are many comments here in this thread.
andyg
Enthusiast
Posts: 58
Liked: 5 times
Joined: Apr 23, 2014 9:51 am
Full Name: Andy Goldschmidt
Contact:

Re: VBR 9.5 - REFS (win2016) - Tips and tricks and gotchas

Post by andyg »

thanks, can you delete this thread or lock it ?
-= VMCE v9 certified =-
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev »

No worries, we merge instead, as this yield to better search results in the future (all leading to the same topic) with everyone using different words to describe the same thing.
adapterer
Expert
Posts: 227
Liked: 46 times
Joined: Oct 12, 2015 11:24 pm
Contact:

Re: VBR 9.5 - REFS

Post by adapterer »

Guys, could we for example seed the data normally to the cloud repository, then do a backup copy to the same repository (or another cloud repository) to obtain the benefits of ReFS cloning?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev » 1 person likes this post

Yes, this should work except you cannot backup copy to the same repository, however you can do this if you create another repository pointing to the same volume. Thanks!
adapterer
Expert
Posts: 227
Liked: 46 times
Joined: Oct 12, 2015 11:24 pm
Contact:

Re: VBR 9.5 - REFS

Post by adapterer »

Another question..

For Cloud Connect, if a client (with 9.5 update of course) who has a local repository using say NTFS, and they do a backup copy to cloud repository with ReFS, does the block cloning still work?
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: VBR 9.5 - REFS

Post by Mike Resseler »

Hi,

Short answer: Yes. Let's say (to keep it simple) the client copies the first (full) and then 6 days of increments to your ReFS cloud repository. After the next incremental, the amount of restore points is exceeded so there will be a synthetic full being created. At that point in time, automatically the block cloning will be utilized even though the client does not have this functionality on his repository.

Brgds,
Mike
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: VBR 9.5 - REFS

Post by dellock6 »

Mike Resseler wrote:Let's say (to keep it simple) the client copies the first (full) and then 6 days of increments to your ReFS cloud repository. After the next incremental, the amount of restore points is exceeded so there will be a synthetic full being created.
Remember we are talking about backup copy, not backups, so there is no synthetic full in it unless the customer either uses a GFS retention (that creates periodic fulls) or the option in a BCJ to make a full manually, Per se BCJ is forever incremental.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
chattingh
Enthusiast
Posts: 57
Liked: 3 times
Joined: Jan 22, 2016 12:45 am
Full Name: Carl Hattingh
Contact:

Re: VBR 9.5 - REFS

Post by chattingh »

The limitation of not being able to leverage ReFS block cloning until an Active Full is taken on the Backup Copy Job is not the best news in instances where there is insufficient bandwidth to perform said Active Full's across a WAN connection. I can see the issue; its around laying the VBK file down with the correct starting & ending alignment if I've understood the doco correctly.

Two questions: Would a File Copy operation through the Veeam console from a temp or staging repo to the destination repo achieve the same outcome as an Active Full (i.e. aligning the VBK correctly), and if not, would Veeam consider creating a utility that can be used for this purpose to aid in seeding BCJ's whilst retaining the goodness of ReFS block cloning?

Luca would you be able to comment on this?

Regards
Carl
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: VBR 9.5 - REFS

Post by Mike Resseler »

Carl,

I don't believe a file copy operation would solve this as it is not "written" the same way as our backup files.

We understand the concern and feel the pain that you have with seeding today. Out of interest... What type of utility would you consider helpful. A simple "windows type of explorer" that gets the seeded information into the database? Or something else?
chattingh
Enthusiast
Posts: 57
Liked: 3 times
Joined: Jan 22, 2016 12:45 am
Full Name: Carl Hattingh
Contact:

Re: VBR 9.5 - REFS

Post by chattingh »

Thanks for that Mike. I was just thinking along the lines of a cmd line utility.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: VBR 9.5 - REFS

Post by dellock6 »

Carl, you are correct any type of file copy done at the OS level is not "refs blockcloning" aware, so the resulting file will not be aligned.
So the only possible solution is to have such a tool that can do those refs-aware copies. A backup copy job using the received file as a seed would do the trick, but BCJ still needs to use the infrastructure information as a source to create the job, even a dummy job. This is hard to do in a service provider environment, but if VBR can see the source infrastructure, it may be worth a test to use backup copy instead of a file copy to seed the refs volume. It's not the best workaround but it may help.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
adapterer
Expert
Posts: 227
Liked: 46 times
Joined: Oct 12, 2015 11:24 pm
Contact:

Re: VBR 9.5 - REFS

Post by adapterer »

A possible process that requires some manual work but would be reasonably efficient:

1. Setup seed disk/NAS as an 'additional' client cloud backup repository
2. Client rescan of resources / import encrypted backups etc
3. Create backup copy job from USB disk / NAS to permanent storage device.
4. Delete this job after complete
5. Create new backup copy job from source backup and map to cloud data

Would this work?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: VBR 9.5 - REFS

Post by dellock6 »

It may succeed as the write to the permanent storage would be done by Veeam datamover, so it would be refs aware. Something I'd need to test...
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
pesos
Expert
Posts: 205
Liked: 17 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: VBR 9.5 - REFS

Post by pesos »

Hi all,

I'm confused as we have a new 2016 setup with 9.5 and an REFS formatted repository (DAS).

I have a default incremental job with synthetic fulls creating on Saturday. It has created two synthetics so far so I have three large VBKs (About 655 gigs each) with the VIBs inbetween.

When I check properties of the folder, it is showing both Size and Size on Disk as 1.83 TB - not seeing any space savings.

Am I doing something wrong? Thanks!
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: VBR 9.5 - REFS

Post by tsightler » 1 person likes this post

As noted previously in this thread, with ReFS and block clone you cannot just look at the properties of the folder as this will not display the space savings since the Windows folder properties dialogue still counts the space for each file even if underneath the blocks are shared via block clone. The only way to really "see" the space savings is to compare what you see in the folder properties to the used space on the volume overall.

Below is an example screenshot:
Image

This screenshot shows a 500GB ReFS repo with ~80GB of remaining free space, even though the stats on the repo folder show that there are currently 592GB of backup files in the folder, which would be impossible without block clone, that's ~172GB of savings. This repo would be out of space if it was NTFS or if it was not using ReFS block clone properly.
pesos
Expert
Posts: 205
Liked: 17 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: VBR 9.5 - REFS

Post by pesos »

Got it - I see now - thanks!
svallance
Service Provider
Posts: 37
Liked: 13 times
Joined: Apr 12, 2016 12:15 pm
Full Name: Shane Vallance
Contact:

Re: VBR 9.5 - REFS

Post by svallance »

dellock6 wrote:Remember we are talking about backup copy, not backups, so there is no synthetic full in it unless the customer either uses a GFS retention (that creates periodic fulls) or the option in a BCJ to make a full manually, Per se BCJ is forever incremental.
So if GFS is enabled on the backup copy job the next time it creates a synthetic GFS full would the backup files then start to see the benefits of REFS? Or will it be necessary to create a new active full or reseed?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev »

svallance wrote:So if GFS is enabled on the backup copy job the next time it creates a synthetic GFS full would the backup files then start to see the benefits of REFS?
This, although fast cloning will kick in from the second synthetic GFS only. You will see the corresponding tag in the job action log when it does.

For fast cloning to work, ALL backup files involved in creating the synthetic full must be created by 9.5 job on ReFS 3.1 repository. As long as at least one file does not meet this requirement, for example it was created with 9.0 and moved to ReFS repository manually, fast cloning will not work.
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: VBR 9.5 - REFS

Post by lando_uk »

In 2016, Is there not a powershell cmd or GUI that will tell you how much disk has been saved due to Cloning, like with the deduplication feature? That would be good, as then the Veeam Console Backup Repositories list could maybe show it as a new column.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: VBR 9.5 - REFS

Post by tsightler »

It might be there, but I haven't been able to find it. There's is a drastic difference in the way the deduplication is accounted for compared to block cloning because they operate very differently. I can't even figure out a way to tell how many blocks are cloned between two files.
tom.starren
Enthusiast
Posts: 26
Liked: 1 time
Joined: Dec 03, 2014 12:31 pm
Full Name: Tom Starren
Contact:

[MERGED] ReFS or deduplication in Windows Server 2016

Post by tom.starren »

Hello,

Currently i'm creating a business case to upgrade and improve our current VEEAM environment. I first want to migrate from HPE DAS storage (with multiple RAID sets combined with storage spaces in Windows 2012 R2) to JBOD storage completely managed by storage spaces. First question here, does anybody have a rule on how many parity disks, etc. you should have with 24 or 60 JBOD disks?

The JBOD storage will be connected to a server which is running Windows Server 2016 (currently we are running Windows 2012 R2 with deduplication enabled), but I don't know if I should go with ReFS file system or deduplication (as far as I know we cannot use both at this point). With ReFS we can do great things, but I don't know what it does with the current disk savings we've got. With the current setup it's hard to predict when the physical storage will be full and when expansion is needed.

I hope someone can advice me on this.

Kr,
Tom
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: VBR 9.5 - REFS

Post by lando_uk »

tsightler wrote:It might be there, but I haven't been able to find it. There's is a drastic difference in the way the deduplication is accounted for compared to block cloning because they operate very differently. I can't even figure out a way to tell how many blocks are cloned between two files.
With this lack of explorer or powershell integration, it almost feels like a feature still in beta - Unlike other features that get enhancements and bug fixes along the way from rollups, new versions of ReFS would surely need a format. I guess what I'm trying to say, is it tested and mature enough to invest £100K on new servers with S2D, its very cutting edge to implement across all backups - Maybe for now, it might be wise for just copy jobs and archive, rather than primary.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 212 guests