Availability for the Always-On Enterprise
Post Reply
antipolis
Enthusiast
Posts: 68
Liked: 9 times
Joined: Oct 26, 2016 9:17 am
Contact:

Re: VBR 9.5 - REFS

Post by antipolis » Dec 01, 2016 9:04 pm

this is what i had in mind... monitoring the refs notifications then either trigger an active full or backup health check when needed

Gostev
Veeam Software
Posts: 22981
Liked: 2882 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev » Dec 01, 2016 9:45 pm

Exactly!

push3r
Enthusiast
Posts: 36
Liked: 6 times
Joined: May 17, 2013 11:54 pm
Contact:

Re: VBR 9.5 - REFS

Post by push3r » Dec 01, 2016 9:48 pm

Gostev wrote: Just wanted to note that both the Integrity Streams and our ReFS integration is available for simple ReFS volumes too, they don't have to be backed by Storage Spaces. The only thing you lose without Storage Spaces is bit rot auto-healing from data mirror or parity sets. However, you will still get notified about corruption appearing, which is always good to know in advance and take care of that with an Active Full (instead of finding out about said corruption upon the restore).
Thank you Gostev for clarify some of the terminologies. That's great news regarding Integrity Streams support with simple ReFS volumes.

antipolis
Enthusiast
Posts: 68
Liked: 9 times
Joined: Oct 26, 2016 9:17 am
Contact:

Re: VBR 9.5 - REFS

Post by antipolis » Dec 01, 2016 9:49 pm 1 person likes this post

so with SS out of the picture it all comes down to :

ntfs for deduplication
or
refs for integrity checks + fast and space efficient synthetics/GFS backups

ReFS just sounds like it brings more to the table

push3r
Enthusiast
Posts: 36
Liked: 6 times
Joined: May 17, 2013 11:54 pm
Contact:

Re: VBR 9.5 - REFS

Post by push3r » Dec 01, 2016 9:52 pm 1 person likes this post

look at my recommendation. :)

NTFS with dedup for regular Backup Job repo (forever incremental with monthly full). ReFS won't do anything here as nothing is being synthesized.

ReFS for Backup Copy Job with GFS policy.

As simple as that and it's what I have just implemented in my environment.

antipolis
Enthusiast
Posts: 68
Liked: 9 times
Joined: Oct 26, 2016 9:17 am
Contact:

Re: VBR 9.5 - REFS

Post by antipolis » Dec 01, 2016 10:00 pm

yes I agree with your points but I just meant that in the case you can't (or do not want) to run active fulls regularly maybe it's better to use refs integrity checks and run active full only when needed

nmdange
Expert
Posts: 341
Liked: 82 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: VBR 9.5 - REFS

Post by nmdange » Dec 01, 2016 11:46 pm

push3r wrote:
look at my recommendation. :)

NTFS with dedup for regular Backup Job repo (forever incremental with monthly full). ReFS won't do anything here as nothing is being synthesized.

ReFS for Backup Copy Job with GFS policy.

As simple as that and it's what I have just implemented in my environment.
If you are doing forward forever incremental then ReFS will help during the merge process. On one of my backup jobs with large change data the merge process can take hours every day which will go away with ReFS!

You can actually do both using an interesting strategy that is described here: https://charbelnemnom.com/2016/10/how-t ... p-storage/

Basically you have a Hyper-V 2016 host with NTFS, and you run your repository in a VM with ReFS enabled then enable the "Virtualized Backup" Dedupe setting on the host. The post is for DPM 2016 but the concept is the same with Veeam since both are leveraging the same ReFS APIs.

saarq
Lurker
Posts: 1
Liked: 4 times
Joined: Apr 19, 2016 6:55 pm
Full Name: Nate
Contact:

Re: VBR 9.5 - REFS

Post by saarq » Dec 08, 2016 8:22 pm 4 people like this post

I just want to say that this has been a godsend. I have a 4TB file server and the synthetic fulls were taking 80hours and still not completing (yes, the repository performance, which is slow, was the major factor in this.) I was looking at having to get some new hardware when 9.5 was released. Ran my first synthetic full on that job this weekend and it only took 30 minutes.

antipolis
Enthusiast
Posts: 68
Liked: 9 times
Joined: Oct 26, 2016 9:17 am
Contact:

Re: VBR 9.5 - REFS

Post by antipolis » Dec 09, 2016 10:06 am

antipolis wrote:this is what i had in mind... monitoring the refs notifications then either trigger an active full or backup health check when needed
now that I think of it it would be really nice to have this integrated in VBR (or a combination of One/VBR) : when integrity streams detects a corrupted block then an active full is triggered

lando_uk
Expert
Posts: 280
Liked: 21 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: VBR 9.5 - REFS

Post by lando_uk » Dec 09, 2016 12:06 pm

On this subject.

A typical Veeam Backup Validator task takes me about 8 hrs to validate a single job with 30 VMs / 3TB, this is because of slow RAID6. Would ReFS also speed up this health check? Because if it does, then that would be a big deal.

Gostev
Veeam Software
Posts: 22981
Liked: 2882 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR 9.5 - REFS

Post by Gostev » Dec 09, 2016 3:24 pm

No, it will make zero difference. Validator needs to read the entire backup file contents regardless of the file system to calculate its checksum. The bigger question is if you need Validator at all considering the presence of data integrity streams (see the corresponding discussion on the previous page of this topic).

ds2
Enthusiast
Posts: 80
Liked: 18 times
Joined: Jul 16, 2015 6:31 am
Full Name: Rene Keller
Contact:

Re: VBR 9.5 - REFS

Post by ds2 » Dec 13, 2016 8:56 am

Are there also REFs-Benefits when using Foreverforward to a REFS-Target? What about the merge, is it faster by pointing only to exiting blocks? Or are there only benefits by GFS or syntetic fulls?

Greetings

Ds2

v.Eremin
Veeam Software
Posts: 15140
Liked: 1141 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VBR 9.5 - REFS

Post by v.Eremin » Dec 13, 2016 9:02 am 1 person likes this post

Two main benefits provided by ReFS are space savings and merge/synthetic performance increase. So, answering your question, yes, merge process will be much faster, should you stick to ReFS target. Thanks.

ds2
Enthusiast
Posts: 80
Liked: 18 times
Joined: Jul 16, 2015 6:31 am
Full Name: Rene Keller
Contact:

Re: VBR 9.5 - REFS

Post by ds2 » Dec 13, 2016 9:05 am

Thanks for clarification.

Amarokada
Novice
Posts: 7
Liked: 1 time
Joined: Jan 30, 2015 4:24 pm
Full Name: Rob Perry
Contact:

[MERGED] Windows 2016/ReFS magic, how do I know it's working

Post by Amarokada » Dec 19, 2016 9:57 am

Hi All

I've recently done an in-place Windows 2012 R2 to Windows 2016 upgrade on all my Veeam servers, and upgraded Veeam to v9.5. My aim is to utilise the new ReFS block pointer efficiencies so that synthetic fulls don't take all day. Under Windows 2012 R2 my jobs were using full weekly backups with Windows Dedupe crunching them down, but with ReFS I'm expecting a similar deduplication effect as part of the file block linking.

As part of the upgrade, I deleted all local backups (from within the Veeam interface "disk" section) to clear out the historical database. I then formatted the local RAID disk with ReFS (4k allocation size), and then kicked off active-full backups of all my jobs.

All my Veeam servers save their backup files to a local RAID disk (via a local repository setup within a local scale-out repository, so I can add further disks in the future without having to change the destinations in the jobs).

This weekend saw the first synthetic full triggered, and I'm not seeing what I expected I would.

One job has about 3.7TB of backup files to crunch through (1 active-full followed by 3 incrementals). The repository is configured to save each VM as a separate file.
It is still running now and has been for 2 hours and is only 39% through the synthetic full creation. Taskmanager shows the disk at 100% active time with about 200MB/sec reading and writing going on.

Also another job which has finished (took 55 minutes to make the synthetic fulls of 11 VMs), had 396GB of backup files (1 active-full followed by 4 incrementals). The resulting 372GB of synthetic fulls are shown as taking 372GB of disk space when viewing the file properties. The whole backup folder shows 768GB size, and 768GB size on disk in folder properties.

I was under the impression that quite a bit of the synthetic fulls would show up as taking no disk space as the matched blocks would just be pointers in the existing backup files.

One thing to note, I have the repository settings set to allow 8 concurrent tasks. However I don't think that is an issue since the 3.7TB job is the only one running now, and still the disk is at 100% active time.

So with the above information is it possible to know if ReFS is doing its thing right? and if not, what step did I miss when reformatting the disk?

Cheers
Rob

Post Reply

Who is online

Users browsing this forum: patrick.shea, rickrbyrne, vhernandez and 40 guests