Comprehensive data protection for all workloads
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Veeam Next Big Thing - REFS

Post by mkretzer »

Hello,

will the new REFS functionality provide any benefit for ACTIVE FULL backups? Or will they just take all the space as before?

Markus
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff »

Active fulls, no since they go out and read off the production environment and actively write out a full backup file. The ReFS enhancements are obviously specific to 9.5 and having a Server 2016 Repository formatted with ReFS and will be seen on any of the transform operations - GFS, Reverse Incremental and Synthetics Fulls.
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer »

Reverse incrementals? How will it help there? Are the new blocks injected from the incremental files by REFS?
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff »

The full restore point is always latest on disk with Reverse Incremental backup mode. So during this process the changed blocks of data read from the production environment are injected into the VBK. The replaced blocks are read and written out to a resulting .VRB file. During that process that the BlockClone API would be leveraged resulting in much less IO as well as space usage on the ReFS repository.
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer » 1 person likes this post

This is sooooo cool and i believe the best feature of 9.5. Why in the world did they only take 10 seconds for that in the presentation but talk forever about stuff noone wants to know??

Can you also tell us about the backup/restore acceleration technologies? Why should it reduce load on the primary storage?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy » 1 person likes this post

Some details can be found in the corresponding breakout session.
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer »

Sadly no. There are infos about the processing engine enhancements but not about the backup/restore acceleration technologies...
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » 1 person likes this post

Would the performance enhancements with ReFS also apply to Merge operations with Forward Forever Incrementals?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Veeam Next Big Thing - REFS

Post by dellock6 »

Yes, as only metadata needs to be updated, without any real block movement on the disks.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy »

mkretzer wrote:Sadly no. There are infos about the processing engine enhancements but not about the backup/restore acceleration technologies...
You can actually get some regarding backup/restore acceleration as well: intelligent source VM data fetching and custom-tailored approach to reading data from different types of the backup storage during the restore (these are mostly under-the-hood code optimizations, so not much to enlarge on them), broker service, direct restore from tape.
crazzyeddie
Enthusiast
Posts: 29
Liked: 4 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie »

And ReFS support will require Server 2016. No support for 2012R2?
DaveWatkins
Veteran
Posts: 370
Liked: 97 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by DaveWatkins »

As far as I understand it, all these new features are only available in Server 2016 because it has ReFS 1.3, which isn't available in previous Windows versions
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Veeam Next Big Thing - REFS

Post by mkretzer »

Can an REFS volume be upgraded to the new version?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam Next Big Thing - REFS

Post by veremin » 1 person likes this post

And ReFS support will require Server 2016. No support for 2012R2?
Correct.
Can an REFS volume be upgraded to the new version?
You'd better ask this question on MS community whether it'd possible to update existing volumes to new ReFS version during 2012 (R2) -> 2016 upgrade.

Thanks.
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: Veeam Next Big Thing - REFS

Post by vClintWyckoff »

Well its really a best practice to not upgrade operating system versions (2012R2->2016) and rebuild on the newest OS version. However, if you're talking about 'upgrading' Hyper-V hosts MS can in fact perform zero downtime, rolling cluster upgrades.
Can an REFS volume be upgraded to the new version?
As of TP5 the version of ReFS is not updated when upgrading from previous OS versions. Not sure what will make it into RTM, so we'll just have to wait and see.
crazzyeddie
Enthusiast
Posts: 29
Liked: 4 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie » 1 person likes this post

vClintWyckoff wrote:Well its really a best practice to not upgrade operating system versions (2012R2->2016) and rebuild on the newest OS version.
It sounds like Microsoft took a page out of the Windows 10 update book and are now "recommending" the 2012 to 2016 upgrade as an in-place install, not rebuild.
gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 »

I did a quick fag packet calculation and according to the info shared in the breakout session I could be looking at backups currently using around 46TB of space going down to around 20TB.

I noticed in the breakout session the backup option repository option to split VMs into different files was required. Will backups with ReFS repository support Veeam compression and deduplication and multiple VMs per backup file?

Would be a shame if we get the promise of zero space synthetic backups only to have a load of caveats with the tech.
crazzyeddie
Enthusiast
Posts: 29
Liked: 4 times
Joined: Nov 11, 2014 9:01 pm
Full Name: Will
Contact:

Re: Veeam Next Big Thing - REFS

Post by crazzyeddie »

gary.martin7 wrote:Would be a shame if we get the promise of zero space synthetic backups only to have a load of caveats with the tech.
Agreed. What about encryption? That's extremely important for many of us.
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Next Big Thing - REFS

Post by Gostev »

I am not aware of any functionality limitations at all.
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange »

So since ReFS still won't support Dedup in Windows Server 2016 (I think), I guess we'll have to pick between getting to use the better dedupe in 2016 or all the other benefits with ReFS :(
gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 » 1 person likes this post

I too am using Windows 2012 R2 Dedupe for Veeam repositories. The Scale Out repositories helped there because I split full backups and incrementals to different drives and only dedupe the fulls disk. However, even though the space is reduced using this method it does load up the server (I process the data outside of the backup window) when the dedupe is in progress. You also have to take care of garbage collection when you delete files.

I see it like this.

Dedupe folds the fulls (and incrementals I guess) on top of each other to remove redundant data (seen it before). This is post process and takes time to achieve. Also not great if any single full is over 1TB. It is very good when the same data exists in multiple backups on the same disk.
ReFS is pretty much doing the same thing for synthetic fulls only, but rather than removing redundant data it never creates it (I got the blocks, I'll just pretend it is a new file). Where you loose here is when your backup files contain the same blocks across files. Where you gain is very fast and io light creation of synthetic fulls.

I guess what you need to do is leverage as much of Veeam's own deduplication in the backup job (bundle like machines together) then you can get the best of both with ReFS. < This statement very much open to discussion.
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange »

For most of my backup jobs, there is only 1 full backup on disk with a number of forward and/or reverse incremental files to meet the short-term retention requirements (usually 14 days to 60 days). In this case, dedupe either using Windows or an appliance doesn't buy that much because there is minimal duplication already. Dedupe makes the most sense when you have long term retention requirements and you are using GFS with Backup Copy jobs.

I suspect if you have a bunch of full backups created via GFS, that ReFS would not provide the same benefits. So perhaps to achieve the best setup, using ReFS on most repositories but then having a separate NTFS formatted repository with dedupe enabled to hold long-term GFS backups would be the best option?
gary.martin7
Novice
Posts: 7
Liked: 1 time
Joined: Aug 31, 2016 1:30 pm
Full Name: Gary Martin
Contact:

Re: Veeam Next Big Thing - REFS

Post by gary.martin7 »

I have a couple of retention policies configured here in my environment. I have a 14 day backup for things like app servers that does run from a single full and I agree that no help would come from ReFS. Also, dedupe is not good here as the incremental rolling into the full nightly would mean at lot of processing to keep up with the changes. However we also have a policy for backing up fileservers and sql servers which has a 90 day retention. I find that extending increments past about a month results in a long recovery time (has to read full then every incremental in the chain). As ReFS synthetic backups are just pointers to existing blocks this means that you can introduce fulls at any time at no storage cost which should have the result of speeding up recovery times (instant recovery should benefit too).

Not sure what you mean by GFS, not a term I have come across before. I do use Backup Copy jobs, but these are set to 2 restore point (minimum available) in order to stage for tape backup.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Veeam Next Big Thing - REFS

Post by dellock6 » 2 people like this post

Yes Veeam "source" deduplication and compression is always there, so you get the benefits regardless which storage you use.
Then, I see to be honest ReFS even better potentially than deduplication. As it has been said, why do I want to waste IO to first write and then delete with deduplication a block? Way better to never store it in the first place when it comes to create synthetic backups.

On the topic of data reduction, whatever type of job you are running, the savings are all going to be there, let me explain:
- any incremental backup during a day has almost unique data, so chances to be reduced by deduplication are lower. Dedupe will not help here just like ReFS will not help
- when you reach the retention however, there will be a transform operation, and this is the place where ReFS will shine. Instead of rehydrating a full to inject the new blocks ReFS will just write new metadata in his filetable. I see this scenario better for ReFS than deduplication
- GFS: same thing. At every interval, Veeam has to create the new full. In the dedupe appliance, again the file has to be created for real, while in ReFS it's just a metadata update. And since blocks are not copied, they don't waste space exactly like in the deduplication scenario, with the benefit of lower IO as deduplication will have to read and write tons of blocks to create the new full

I'm honestly wondering if deduplication will be needed AT ALL with this new technology, especially when you add to the discussion the "restore" performance. ReFS is not deduped, so there's nothing to re-hydrate during the restore.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Veeam Next Big Thing - REFS

Post by nmdange » 3 people like this post

@gary GFS refers to long term retention on disk with Backup Copy jobs by keeping weekly, monthly, yearly backups. Since each of these is actually a full backup the disk space requirements are quite large. https://helpcenter.veeam.com/backup/hyp ... y_gfs.html

@Luca
I understand how ReFS helps with forward forever incremental. When the oldest incremental is merged into the full backup file, instead of copying the data from the vib file into the vbk file, you can just update the metadata in the vbk file to point to the same physical blocks as the vib file.

If you are saying this also applies when you have multiple synthetic fulls with GFS, then this new ReFS functionality is even better than I thought!

The only argument for dedupe would be if you could get additional space savings when deduplicating between multiple backup jobs (or if you have per-vm backup files). However if you group VMs that tend to have similar data into the same job, there would be very little commonality between jobs anyway. With all the performance issues, overhead, cost, etc. of having to use a dedupe appliance, I see very little reason to use them at all. It's also another reason to stay away from NAS devices. I've always thought a Windows server with a ton of cheap internal storage was the best choice for backup repositories and this just confirms that :)
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Next Big Thing - REFS

Post by Gostev »

nmdange wrote:If you are saying this also applies when you have multiple synthetic fulls with GFS, then this new ReFS functionality is even better than I thought!

The only argument for dedupe would be if you could get additional space savings when deduplicating between multiple backup jobs (or if you have per-vm backup files). However if you group VMs that tend to have similar data into the same job, there would be very little commonality between jobs anyway. With all the performance issues, overhead, cost, etc. of having to use a dedupe appliance, I see very little reason to use them at all. It's also another reason to stay away from NAS devices. I've always thought a Windows server with a ton of cheap internal storage was the best choice for backup repositories and this just confirms that :)
Ditto. You are spot on.
ds2
Enthusiast
Posts: 82
Liked: 19 times
Joined: Jul 16, 2015 6:31 am
Full Name: Rene Keller
Contact:

Re: Veeam Next Big Thing - REFS

Post by ds2 »

Hello,

does it also work for Endpoint Jobs and Endpoit Copy Jobs when target is a B&R Repo with REFS?

Greetings

ds2
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: Veeam Next Big Thing - REFS

Post by skrause »

Will the ReFS support in 9.5 work with ReFS on 2012R2 or will it require an update of components to 2016?

The ReFS support sounds cool and I would like to do some testing, especially on copy jobs, but I likely won't be able to move our infrastructure to 2016 until the middle of next year at the earliest.
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam Next Big Thing - REFS

Post by foggy »

As mentioned above, ReFS support will require Windows Server 2016.
szwicker
Service Provider
Posts: 137
Liked: 1 time
Joined: Jan 08, 2010 5:15 pm
Full Name: Seth Zwicker
Contact:

Re: Veeam Next Big Thing - REFS

Post by szwicker »

And the benefits of Veeam9.5/REFS3.0/Win2016 WILL also take place on the Cloud Connector side?
So when I have those big offsite copy jobs running (which are basically like forever incrementals and take forever( I'll get the performance boost as well?
I'm asking because I'm contemplating putting dual Quantum appliances and letting them dedupe and replicate.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 206 guests