Comprehensive data protection for all workloads
mkretzer
Expert
Posts: 571
Liked: 129 times
Joined: Dec 17, 2015 7:17 am
Contact:

Veeam Next Big Thing - REFS

Post by mkretzer » Aug 23, 2016 10:46 pm

Hello,

will the new REFS functionality provide any benefit for ACTIVE FULL backups? Or will they just take all the space as before?

Markus

vClintWyckoff
Expert
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff » Aug 23, 2016 10:58 pm

Active fulls, no since they go out and read off the production environment and actively write out a full backup file. The ReFS enhancements are obviously specific to 9.5 and having a Server 2016 Repository formatted with ReFS and will be seen on any of the transform operations - GFS, Reverse Incremental and Synthetics Fulls.

mkretzer
Expert
Posts: 571
Liked: 129 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer » Aug 23, 2016 11:08 pm

Reverse incrementals? How will it help there? Are the new blocks injected from the incremental files by REFS?

vClintWyckoff
Expert
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff » Aug 24, 2016 3:52 am

The full restore point is always latest on disk with Reverse Incremental backup mode. So during this process the changed blocks of data read from the production environment are injected into the VBK. The replaced blocks are read and written out to a resulting .VRB file. During that process that the BlockClone API would be leveraged resulting in much less IO as well as space usage on the ReFS repository.

mkretzer
Expert
Posts: 571
Liked: 129 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer » Aug 24, 2016 5:28 am 1 person likes this post

This is sooooo cool and i believe the best feature of 9.5. Why in the world did they only take 10 seconds for that in the presentation but talk forever about stuff noone wants to know??

Can you also tell us about the backup/restore acceleration technologies? Why should it reduce load on the primary storage?

foggy
Veeam Software
Posts: 18439
Liked: 1588 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy » Aug 24, 2016 5:13 pm 1 person likes this post

Some details can be found in the corresponding breakout session.

mkretzer
Expert
Posts: 571
Liked: 129 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer » Aug 24, 2016 7:05 pm

Sadly no. There are infos about the processing engine enhancements but not about the backup/restore acceleration technologies...

nmdange
Expert
Posts: 478
Liked: 119 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » Aug 24, 2016 10:08 pm 1 person likes this post

Would the performance enhancements with ReFS also apply to Merge operations with Forward Forever Incrementals?

dellock6
Veeam Software
Posts: 5787
Liked: 1655 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Veeam Next Big Thing - REFS

Post by dellock6 » Aug 24, 2016 10:13 pm

Yes, as only metadata needs to be updated, without any real block movement on the disks.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

foggy
Veeam Software
Posts: 18439
Liked: 1588 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy » Aug 25, 2016 11:56 am

mkretzer wrote:Sadly no. There are infos about the processing engine enhancements but not about the backup/restore acceleration technologies...
You can actually get some regarding backup/restore acceleration as well: intelligent source VM data fetching and custom-tailored approach to reading data from different types of the backup storage during the restore (these are mostly under-the-hood code optimizations, so not much to enlarge on them), broker service, direct restore from tape.

crazzyeddie
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie » Aug 28, 2016 11:36 pm

And ReFS support will require Server 2016. No support for 2012R2?

DaveWatkins
Expert
Posts: 349
Liked: 94 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by DaveWatkins » Aug 29, 2016 3:13 am

As far as I understand it, all these new features are only available in Server 2016 because it has ReFS 1.3, which isn't available in previous Windows versions

mkretzer
Expert
Posts: 571
Liked: 129 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer » Aug 29, 2016 7:19 am

Can an REFS volume be upgraded to the new version?

veremin
Product Manager
Posts: 17047
Liked: 1470 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam Next Big Thing - REFS

Post by veremin » Aug 29, 2016 8:19 am 1 person likes this post

And ReFS support will require Server 2016. No support for 2012R2?
Correct.
Can an REFS volume be upgraded to the new version?
You'd better ask this question on MS community whether it'd possible to update existing volumes to new ReFS version during 2012 (R2) -> 2016 upgrade.

Thanks.

vClintWyckoff
Expert
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff » Aug 29, 2016 5:00 pm

Well its really a best practice to not upgrade operating system versions (2012R2->2016) and rebuild on the newest OS version. However, if you're talking about 'upgrading' Hyper-V hosts MS can in fact perform zero downtime, rolling cluster upgrades.
Can an REFS volume be upgraded to the new version?
As of TP5 the version of ReFS is not updated when upgrading from previous OS versions. Not sure what will make it into RTM, so we'll just have to wait and see.

crazzyeddie
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie » Aug 29, 2016 8:44 pm 1 person likes this post

vClintWyckoff wrote:Well its really a best practice to not upgrade operating system versions (2012R2->2016) and rebuild on the newest OS version.
It sounds like Microsoft took a page out of the Windows 10 update book and are now "recommending" the 2012 to 2016 upgrade as an in-place install, not rebuild.

gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 » Aug 31, 2016 2:52 pm

I did a quick fag packet calculation and according to the info shared in the breakout session I could be looking at backups currently using around 46TB of space going down to around 20TB.

I noticed in the breakout session the backup option repository option to split VMs into different files was required. Will backups with ReFS repository support Veeam compression and deduplication and multiple VMs per backup file?

Would be a shame if we get the promise of zero space synthetic backups only to have a load of caveats with the tech.

crazzyeddie
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie » Aug 31, 2016 10:14 pm

gary.martin7 wrote:Would be a shame if we get the promise of zero space synthetic backups only to have a load of caveats with the tech.
Agreed. What about encryption? That's extremely important for many of us.

Gostev
SVP, Product Management
Posts: 25082
Liked: 3668 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Next Big Thing - REFS

Post by Gostev » Aug 31, 2016 11:58 pm

I am not aware of any functionality limitations at all.

nmdange
Expert
Posts: 478
Liked: 119 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » Sep 01, 2016 3:50 am

So since ReFS still won't support Dedup in Windows Server 2016 (I think), I guess we'll have to pick between getting to use the better dedupe in 2016 or all the other benefits with ReFS :(

gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 » Sep 01, 2016 8:09 am 1 person likes this post

I too am using Windows 2012 R2 Dedupe for Veeam repositories. The Scale Out repositories helped there because I split full backups and incrementals to different drives and only dedupe the fulls disk. However, even though the space is reduced using this method it does load up the server (I process the data outside of the backup window) when the dedupe is in progress. You also have to take care of garbage collection when you delete files.

I see it like this.

Dedupe folds the fulls (and incrementals I guess) on top of each other to remove redundant data (seen it before). This is post process and takes time to achieve. Also not great if any single full is over 1TB. It is very good when the same data exists in multiple backups on the same disk.
ReFS is pretty much doing the same thing for synthetic fulls only, but rather than removing redundant data it never creates it (I got the blocks, I'll just pretend it is a new file). Where you loose here is when your backup files contain the same blocks across files. Where you gain is very fast and io light creation of synthetic fulls.

I guess what you need to do is leverage as much of Veeam's own deduplication in the backup job (bundle like machines together) then you can get the best of both with ReFS. < This statement very much open to discussion.

nmdange
Expert
Posts: 478
Liked: 119 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » Sep 01, 2016 1:36 pm

For most of my backup jobs, there is only 1 full backup on disk with a number of forward and/or reverse incremental files to meet the short-term retention requirements (usually 14 days to 60 days). In this case, dedupe either using Windows or an appliance doesn't buy that much because there is minimal duplication already. Dedupe makes the most sense when you have long term retention requirements and you are using GFS with Backup Copy jobs.

I suspect if you have a bunch of full backups created via GFS, that ReFS would not provide the same benefits. So perhaps to achieve the best setup, using ReFS on most repositories but then having a separate NTFS formatted repository with dedupe enabled to hold long-term GFS backups would be the best option?

gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 » Sep 01, 2016 2:36 pm

I have a couple of retention policies configured here in my environment. I have a 14 day backup for things like app servers that does run from a single full and I agree that no help would come from ReFS. Also, dedupe is not good here as the incremental rolling into the full nightly would mean at lot of processing to keep up with the changes. However we also have a policy for backing up fileservers and sql servers which has a 90 day retention. I find that extending increments past about a month results in a long recovery time (has to read full then every incremental in the chain). As ReFS synthetic backups are just pointers to existing blocks this means that you can introduce fulls at any time at no storage cost which should have the result of speeding up recovery times (instant recovery should benefit too).

Not sure what you mean by GFS, not a term I have come across before. I do use Backup Copy jobs, but these are set to 2 restore point (minimum available) in order to stage for tape backup.

dellock6
Veeam Software
Posts: 5787
Liked: 1655 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Veeam Next Big Thing - REFS

Post by dellock6 » Sep 01, 2016 5:21 pm 2 people like this post

Yes Veeam "source" deduplication and compression is always there, so you get the benefits regardless which storage you use.
Then, I see to be honest ReFS even better potentially than deduplication. As it has been said, why do I want to waste IO to first write and then delete with deduplication a block? Way better to never store it in the first place when it comes to create synthetic backups.

On the topic of data reduction, whatever type of job you are running, the savings are all going to be there, let me explain:
- any incremental backup during a day has almost unique data, so chances to be reduced by deduplication are lower. Dedupe will not help here just like ReFS will not help
- when you reach the retention however, there will be a transform operation, and this is the place where ReFS will shine. Instead of rehydrating a full to inject the new blocks ReFS will just write new metadata in his filetable. I see this scenario better for ReFS than deduplication
- GFS: same thing. At every interval, Veeam has to create the new full. In the dedupe appliance, again the file has to be created for real, while in ReFS it's just a metadata update. And since blocks are not copied, they don't waste space exactly like in the deduplication scenario, with the benefit of lower IO as deduplication will have to read and write tons of blocks to create the new full

I'm honestly wondering if deduplication will be needed AT ALL with this new technology, especially when you add to the discussion the "restore" performance. ReFS is not deduped, so there's nothing to re-hydrate during the restore.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

nmdange
Expert
Posts: 478
Liked: 119 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » Sep 01, 2016 7:15 pm 3 people like this post

@gary GFS refers to long term retention on disk with Backup Copy jobs by keeping weekly, monthly, yearly backups. Since each of these is actually a full backup the disk space requirements are quite large. https://helpcenter.veeam.com/backup/hyp ... y_gfs.html

@Luca
I understand how ReFS helps with forward forever incremental. When the oldest incremental is merged into the full backup file, instead of copying the data from the vib file into the vbk file, you can just update the metadata in the vbk file to point to the same physical blocks as the vib file.

If you are saying this also applies when you have multiple synthetic fulls with GFS, then this new ReFS functionality is even better than I thought!

The only argument for dedupe would be if you could get additional space savings when deduplicating between multiple backup jobs (or if you have per-vm backup files). However if you group VMs that tend to have similar data into the same job, there would be very little commonality between jobs anyway. With all the performance issues, overhead, cost, etc. of having to use a dedupe appliance, I see very little reason to use them at all. It's also another reason to stay away from NAS devices. I've always thought a Windows server with a ton of cheap internal storage was the best choice for backup repositories and this just confirms that :)

Gostev
SVP, Product Management
Posts: 25082
Liked: 3668 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Next Big Thing - REFS

Post by Gostev » Sep 01, 2016 9:16 pm

nmdange wrote:If you are saying this also applies when you have multiple synthetic fulls with GFS, then this new ReFS functionality is even better than I thought!

The only argument for dedupe would be if you could get additional space savings when deduplicating between multiple backup jobs (or if you have per-vm backup files). However if you group VMs that tend to have similar data into the same job, there would be very little commonality between jobs anyway. With all the performance issues, overhead, cost, etc. of having to use a dedupe appliance, I see very little reason to use them at all. It's also another reason to stay away from NAS devices. I've always thought a Windows server with a ton of cheap internal storage was the best choice for backup repositories and this just confirms that :)
Ditto. You are spot on.

ds2
Enthusiast
Posts: 82
Liked: 18 times
Joined: Jul 16, 2015 6:31 am
Full Name: Rene Keller
Contact:

Re: Veeam Next Big Thing - REFS

Post by ds2 » Sep 02, 2016 5:13 am

Hello,

does it also work for Endpoint Jobs and Endpoit Copy Jobs when target is a B&R Repo with REFS?

Greetings

ds2

skrause
Expert
Posts: 444
Liked: 93 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: Veeam Next Big Thing - REFS

Post by skrause » Sep 02, 2016 4:45 pm

Will the ReFS support in 9.5 work with ReFS on 2012R2 or will it require an update of components to 2016?

The ReFS support sounds cool and I would like to do some testing, especially on copy jobs, but I likely won't be able to move our infrastructure to 2016 until the middle of next year at the earliest.
Steve Krause
Veeam Certified Architect

foggy
Veeam Software
Posts: 18439
Liked: 1588 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy » Sep 02, 2016 4:48 pm

As mentioned above, ReFS support will require Windows Server 2016.

szwicker
Service Provider
Posts: 126
Liked: 1 time
Joined: Jan 08, 2010 5:15 pm
Full Name: Seth Zwicker
Contact:

Re: Veeam Next Big Thing - REFS

Post by szwicker » Sep 05, 2016 1:49 am

And the benefits of Veeam9.5/REFS3.0/Win2016 WILL also take place on the Cloud Connector side?
So when I have those big offsite copy jobs running (which are basically like forever incrementals and take forever( I'll get the performance boost as well?
I'm asking because I'm contemplating putting dual Quantum appliances and letting them dedupe and replicate.

Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 17 guests