VBR 9.5 - REFS

[JoshuaPostSAMC]

PTide, would I see increase performance iSCSI vs CIFS in general, or only when paired with ReFS?

[Gostev]

Only when paired with ReFS.

[chattingh]

I have a question about how to seed a backup copy job with an ReFS repository. If I understand correctly from this thread, the Fast Clone advantages of an ReFS repo can only be leveraged once the new VBK files are properly block aligned with the filesystem. To achieve that, an active or synthetic full must be run against the ReFS repository. Is my understanding from reading this thread correct?

That implies to me that simply copying existing backup files to an ReFS repo will not activate the Fast Clone API. How then, would one seed a backup copy job to an ReFS repo (for Cloud Connect as an example) whilst obtaining the benefits of the Fast Clone API?

Apologies if this is in the doco, but I coudn't find it.

[vClintWyckoff]

Carl-
Correct, you would need to kick off an active full or a synthetic full to align with the file system to begin to see the benefits. Alternatively, you could do a Backup Copy job with a 1 to 1 match of the original job and after that completes then remap to the newly created backup files.

[chattingh]

Thanks Clint

To confirm, in the scenario where a Backup Copy job is going over the WAN or Internet, and there is insufficient bandwidth to perform an active full, but instead a data seed is required, are we saying that it is not possible to leverage the Fast Clone API?

We typically do the data seed for clients by targeting the Backup Copy job to removable media or NAS, take it to the destination, copy the backup files to the proper repo, then update the job with the proper repo and map the backpup files.

Is it possible to perform a synthetic full backup on a backup copy job? I don't think so as there are no options in the Backup Copy job wizard.

[vClintWyckoff]

To confirm, in the scenario where a Backup Copy job is going over the WAN or Internet, and there is insufficient bandwidth to perform an active full, but instead a data seed is required, are we saying that it is not possible to leverage the Fast Clone API?

We typically do the data seed for clients by targeting the Backup Copy job to removable media or NAS, take it to the destination, copy the backup files to the proper repo, then update the job with the proper repo and map the backpup files.

In this case the customer wouldn't benefit from any [fast merge] operations but any additional jobs that would be run locally against that ReFS volume would benefit. Depending on the external volume, what I have found is that some permit the formatting with ReFS, some do not - but when you do the backup copy to the external media and create the GFS retention, that is a synthetic operation so in that respect BCJ is eligible to benefit from ReFS, but not when you copy off those backup files and they'll literally be copying the real blocks of data.

[andyg]

Hi Guys

Can anyone using ReFS on Win2016 please paste any war stories, tips and tricks in this thread.

I am about to move to ReFS and any tips, tricks or gotchas would be really good. Any step-by-steps guides would be awesome too !!

Try keep it on topic please.

[vClintWyckoff]

Andy: There are many comments here in this thread.

[andyg]

thanks, can you delete this thread or lock it ?

[Gostev]

No worries, we merge instead, as this yield to better search results in the future (all leading to the same topic) with everyone using different words to describe the same thing.

[adapterer]

Guys, could we for example seed the data normally to the cloud repository, then do a backup copy to the same repository (or another cloud repository) to obtain the benefits of ReFS cloning?

[Gostev]

Yes, this should work except you cannot backup copy to the same repository, however you can do this if you create another repository pointing to the same volume. Thanks!

[adapterer]

Another question..

For Cloud Connect, if a client (with 9.5 update of course) who has a local repository using say NTFS, and they do a backup copy to cloud repository with ReFS, does the block cloning still work?

[Mike Resseler]

Hi,

Short answer: Yes. Let's say (to keep it simple) the client copies the first (full) and then 6 days of increments to your ReFS cloud repository. After the next incremental, the amount of restore points is exceeded so there will be a synthetic full being created. At that point in time, automatically the block cloning will be utilized even though the client does not have this functionality on his repository.

Brgds,
Mike

[dellock6]

Let's say (to keep it simple) the client copies the first (full) and then 6 days of increments to your ReFS cloud repository. After the next incremental, the amount of restore points is exceeded so there will be a synthetic full being created.

Remember we are talking about backup copy, not backups, so there is no synthetic full in it unless the customer either uses a GFS retention (that creates periodic fulls) or the option in a BCJ to make a full manually, Per se BCJ is forever incremental.

[chattingh]

The limitation of not being able to leverage ReFS block cloning until an Active Full is taken on the Backup Copy Job is not the best news in instances where there is insufficient bandwidth to perform said Active Full's across a WAN connection. I can see the issue; its around laying the VBK file down with the correct starting & ending alignment if I've understood the doco correctly.

Two questions: Would a File Copy operation through the Veeam console from a temp or staging repo to the destination repo achieve the same outcome as an Active Full (i.e. aligning the VBK correctly), and if not, would Veeam consider creating a utility that can be used for this purpose to aid in seeding BCJ's whilst retaining the goodness of ReFS block cloning?

Luca would you be able to comment on this?

Regards
Carl

[Mike Resseler]

Carl,

I don't believe a file copy operation would solve this as it is not "written" the same way as our backup files.

We understand the concern and feel the pain that you have with seeding today. Out of interest... What type of utility would you consider helpful. A simple "windows type of explorer" that gets the seeded information into the database? Or something else?

[chattingh]

Thanks for that Mike. I was just thinking along the lines of a cmd line utility.

[dellock6]

Carl, you are correct any type of file copy done at the OS level is not "refs blockcloning" aware, so the resulting file will not be aligned.
So the only possible solution is to have such a tool that can do those refs-aware copies. A backup copy job using the received file as a seed would do the trick, but BCJ still needs to use the infrastructure information as a source to create the job, even a dummy job. This is hard to do in a service provider environment, but if VBR can see the source infrastructure, it may be worth a test to use backup copy instead of a file copy to seed the refs volume. It's not the best workaround but it may help.

[adapterer]

A possible process that requires some manual work but would be reasonably efficient:

1. Setup seed disk/NAS as an 'additional' client cloud backup repository
2. Client rescan of resources / import encrypted backups etc
3. Create backup copy job from USB disk / NAS to permanent storage device.
4. Delete this job after complete
5. Create new backup copy job from source backup and map to cloud data

Would this work?

[dellock6]

It may succeed as the write to the permanent storage would be done by Veeam datamover, so it would be refs aware. Something I'd need to test...

[pesos]

Hi all,

I'm confused as we have a new 2016 setup with 9.5 and an REFS formatted repository (DAS).

I have a default incremental job with synthetic fulls creating on Saturday. It has created two synthetics so far so I have three large VBKs (About 655 gigs each) with the VIBs inbetween.

When I check properties of the folder, it is showing both Size and Size on Disk as 1.83 TB - not seeing any space savings.

Am I doing something wrong? Thanks!

[tsightler]

As noted previously in this thread, with ReFS and block clone you cannot just look at the properties of the folder as this will not display the space savings since the Windows folder properties dialogue still counts the space for each file even if underneath the blocks are shared via block clone. The only way to really "see" the space savings is to compare what you see in the folder properties to the used space on the volume overall.

Below is an example screenshot:


This screenshot shows a 500GB ReFS repo with ~80GB of remaining free space, even though the stats on the repo folder show that there are currently 592GB of backup files in the folder, which would be impossible without block clone, that's ~172GB of savings. This repo would be out of space if it was NTFS or if it was not using ReFS block clone properly.

[pesos]

Got it - I see now - thanks!

[svallance]

Remember we are talking about backup copy, not backups, so there is no synthetic full in it unless the customer either uses a GFS retention (that creates periodic fulls) or the option in a BCJ to make a full manually, Per se BCJ is forever incremental.

So if GFS is enabled on the backup copy job the next time it creates a synthetic GFS full would the backup files then start to see the benefits of REFS? Or will it be necessary to create a new active full or reseed?

[Gostev]

So if GFS is enabled on the backup copy job the next time it creates a synthetic GFS full would the backup files then start to see the benefits of REFS?

This, although fast cloning will kick in from the second synthetic GFS only. You will see the corresponding tag in the job action log when it does.

For fast cloning to work, ALL backup files involved in creating the synthetic full must be created by 9.5 job on ReFS 3.1 repository. As long as at least one file does not meet this requirement, for example it was created with 9.0 and moved to ReFS repository manually, fast cloning will not work.

[lando_uk]

In 2016, Is there not a powershell cmd or GUI that will tell you how much disk has been saved due to Cloning, like with the deduplication feature? That would be good, as then the Veeam Console Backup Repositories list could maybe show it as a new column.

[tsightler]

It might be there, but I haven't been able to find it. There's is a drastic difference in the way the deduplication is accounted for compared to block cloning because they operate very differently. I can't even figure out a way to tell how many blocks are cloned between two files.

[tom.starren]

Hello,

Currently i'm creating a business case to upgrade and improve our current VEEAM environment. I first want to migrate from HPE DAS storage (with multiple RAID sets combined with storage spaces in Windows 2012 R2) to JBOD storage completely managed by storage spaces. First question here, does anybody have a rule on how many parity disks, etc. you should have with 24 or 60 JBOD disks?

The JBOD storage will be connected to a server which is running Windows Server 2016 (currently we are running Windows 2012 R2 with deduplication enabled), but I don't know if I should go with ReFS file system or deduplication (as far as I know we cannot use both at this point). With ReFS we can do great things, but I don't know what it does with the current disk savings we've got. With the current setup it's hard to predict when the physical storage will be full and when expansion is needed.

I hope someone can advice me on this.

Kr,
Tom

[lando_uk]

It might be there, but I haven't been able to find it. There's is a drastic difference in the way the deduplication is accounted for compared to block cloning because they operate very differently. I can't even figure out a way to tell how many blocks are cloned between two files.

With this lack of explorer or powershell integration, it almost feels like a feature still in beta - Unlike other features that get enhancements and bug fixes along the way from rollups, new versions of ReFS would surely need a format. I guess what I'm trying to say, is it tested and mature enough to invest £100K on new servers with S2D, its very cutting edge to implement across all backups - Maybe for now, it might be wise for just copy jobs and archive, rather than primary.