Primary ReFS Backup Repository Space Savings?

Availability for the Always-On Enterprise

Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Mon Nov 14, 2016 12:40 pm

Hello there i've read the Blogpost from Luca Dell'Oca (http://www.virtualtothecore.com/en/wind ... epository/)

and he states following:
NOTE: there is NO space saving on primary backups during transform operations, as the same block is always written once, and it’s only moved from the incremental backup file to the full one. Transform operations in primary jobs are about time saving, not space saving.


when i understand that correctly there is only speed benefit from refs on primary backups? i thought and this is what Veeam promised there is space saving on synthetic fulls too?

last week i've upgraded to Veeam 9.5, than did a inplace upgrade from my 2012r2 server... all fine ... then i created a new ReFS BackupRepository and copied all the old Veeam Backupfiles to this Repository.... in the past i did just forward Inc. forever, as synthetic fulls wheren not applicable because of space issues... every week a syn. full with almost 3TB is too much... so the refs "Spaceless full backup technology" promised a huge step forward....

what i saw now over the weekend, that my 7TB Repository where getting filled completely and backups started too fail... today in the morning i checked what happend, and what i saw first was a backup server with without any ram... some process eaten all memory available at least 32gb... every task worked terribly slow on the server... i was not able to start the vbr console but i've managed to delete some old backup files and after some time i get dropped from the server and couldn't connect for some time ... i had to to something different in the time but after that i managed to loggon back and was greeted with dirty shutdown warning.. the machine bsod'ed (maybe because out-of-ram)

Abnormal Program Termination (Windows bug check, STOP: 0x0000013C (0xFFFFA405955E8080, 0x00000000000000E8, 0x0000000000000000, 0x0000000000000000))


anyway... i'm up and running now.. have space left on the backup repository and tried a controlled synthetic full run now...

i see the "fast clone" full backup creation and yes as luca stated, no space savings... the fullbackup does have the fullsize on the disk, what is very disappointing!
Image

all the space is used:
Image

i'm left now with the simple question: why does "space savings" not work in primary backups? as this would have really changed a lot too!
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby dellock6 » Mon Nov 14, 2016 3:13 pm

Hi,
sorry if the post has created this small confusion, the main difference is between incremental backups (IO savings) versus synthetic operations (IO "AND" space savings). So, it's correct that also primary backups can give you space savings if you do synthetic fulls, exactly like GFS retention on a backup copy job. Sorry for the confusion, I'm editing the post to make it more clear. Thanks for the heads-up.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5052
Liked: 1333 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Mon Nov 14, 2016 3:26 pm

ok then i understood it correctly... the question is now why does my syn. full does not leverage space savings... ?! :roll:

edit:
i did a active full now... but i have to wait till tomorrow to try a synthetic full as Veeam does not do a syn. full now after the active full (or the synfull from testing before...) :)

edit2:
btw. should the refs integration work with "perVM" Backupfiles too?
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby vClintWyckoff » Mon Nov 14, 2016 3:43 pm 1 person likes this post

I even myself had to read Luca's post a few times, but it is 100% accurate. Space savings AND Time Savings will be experienced on the Synthetic operations.

So for me to clarify, for example....

Where you need to look for the disk space savings is within the Used Space on the ReFS volume. Synthetic Fulls do not consume space on ReFS volumes, because they are created by referencing the disk blocks already stored on the disk as a part of full and incremental backup files which were previously created. So a good test of this is to have a primary job create a few synthetic fulls or create a backup copy job that creates a few GFS restore points. Then look at how little the used space on ReFS volumes changes after each new synthetic VBK is created.

So you shouldn't be confused by the fact that Windows Explorer shows the actual size of these VBKs, they're literally normal files BUT they're sharing their blocks with existing files. However, you'll note that the total size of all backup files just does not add up to Used Space on the volume.

Image

Also, to clarify - copied VBK / VIB files to a new ReFS volume will not benefit from the FastMerge / Block Clone API. What happens here is that the BlockClone API requires that the file blocks be aligned to the 4k or 64k boundary. So either an Active Full or Synthetic Full are required to get the benefits of the new integration.
vClintWyckoff
Veeam Software
 
Posts: 401
Liked: 93 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Mon Nov 14, 2016 4:26 pm

The issue here was that even the sum on volme information added up to full space...

The file information, as far as I understood it should also make notice od that... At least with the "size on disk" information... As with when using dedup...

But I will see tomorrow after ah active full... Maybe changing backuprepo and using perVM file broke the mechanisms...
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby dellock6 » Mon Nov 14, 2016 4:57 pm

Stefan, as said before, just moving Veeam files into the new ReFS repository is not enough, as blocks need to be aligned to 4k or 64k cluster size to leverage the ReFS blockclone API. The native OS file explorer doesn't align writes, so the file deployed in the new volume are not aligned. You need a full to align the block and start consuming the api with the following incrementals.

In regards to the images you posted, you need to open the properties of the root volume to check the overall disk space used, files in the file system will always show their file size with no space reduction. The 50GB full will always be shown as 50GB regardless the gfs spaceless full backup. Check the root of the volume V: in your example.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5052
Liked: 1333 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Mon Nov 14, 2016 6:34 pm

Ok i unterstand that... As said i did a active full now... But have to wait for tomorrow as now no new synthetic full can be created, only normal increments gets created...

Anyway, even when existing files are copied and of course no space reduction is done from windows explorer, wouldn't it be possible to "activate" the block clone API (alignment) at least with the next syn.full? What I described in my opening post I thought that would happen but instead I created "normal" synfulls that ate all remaining space...
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby tsightler » Mon Nov 14, 2016 6:44 pm 1 person likes this post

Synthetic full is enough to activate block clone, but you can't clone blocks from the old, unaligned chain to the new, aligned chain, so the first creation of a synthetic on a ReFS enabled repository using backups copied from another repository will require the same amount of space as a traditional synthetic. After that, new synthetics should be able to share blocks with previous synthetics. In your screenshot above it definitely looks like the VBK from 11-14 is sharing blocks with the VBK from 11-12, although it's impossible to tell completely with the screenshot provided as you won't see any savings on the file properties.
tsightler
Veeam Software
 
Posts: 4769
Liked: 1738 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Tue Nov 15, 2016 10:46 am

some files looks like to have leveraged the ReFS API now:

Image

i'm however not totally sure what files... furthermore i haven't done a full, or syn.full on my production store with almost 4TB, that filled the store entirely.

at least the fastcloning API does its job very well :)
Image
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby futureweb » Sun Feb 19, 2017 2:45 am

I am wondering if there is a way to see the acutal file or folder size that is used?

With DEDUP you can see acutal file size and file size, but with REFS, unfortunatly this difference was missed.
As you mentioned, only looking at the drive size itself shows you the real size, so not sure if this is something MS is missing,
or any other possibility to view it per file or folder?

We see quite amazing savings. 13,5 TB and about 7 TB used, while we have 1,3 TB of Endpoint backups,
where I am not even sure if it leverages any savings here, as it's only 1 full and many inc's, or if any fast cloning is happening,
when inc's of endpoints are merged.

Anyway, synthetic full take about 27 minutes on 5,1 TB, which is 3,2 GB per second which sounds pretty good to me. (having it weekly and inc's daily).


Thanks
Patrick
futureweb
Enthusiast
 
Posts: 53
Liked: 5 times
Joined: Thu Sep 03, 2015 12:15 am
Full Name: Patrick

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby millardjk » Mon Feb 20, 2017 4:36 am

It's too bad that Microsoft didn't think the fastclone feature could use some explicit UI reporting for space savings. Having to pull up the volume root to see the actual space consumed versus all the folders for the logical file use is...so 1992.
millardjk
Veeam Vanguard
 
Posts: 93
Liked: 18 times
Joined: Sun Dec 09, 2012 3:50 am
Full Name: Jim Millard

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby nsumner » Mon Feb 27, 2017 8:09 am

What I am wondering is, is there ANY way that I can take advantage of block clone on old files? Any way at all! I'm out of space and want to use block clone to solve the problem but I must be able to use it for my archive!
nsumner
Novice
 
Posts: 5
Liked: never
Joined: Mon Feb 27, 2017 8:06 am
Full Name: Noach Sumner

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby Mike Resseler » Tue Feb 28, 2017 9:35 am

Noach,

What do you mean with your old files? Are you saying the files that are already copied to a ReFS repository but are not aligned? (And please be aware that there are indeed space savings as you can see from above but again, it is not a deduplication technology so space savings really can be very dependent)
Mike Resseler
Veeam Software
 
Posts: 3165
Liked: 362 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby nsumner » Tue Feb 28, 2017 9:46 am

Hi what I mean is I currently have backups on a NTFS disk. I want to create a new REFS disk and move the files from the NTFS to the REFS and wipe out the NTFS (and ultimately add those drives to the REFS).

Right now I'm going to need to do things in a very backwards manner (move to NTFS with Dedup). Move to a larger REFS after eliminating the main NTFS drive (I'm buying more HDs so I can create another RAID array during the move). But even after all is said and done I will have what is nearly 50TB of data taking up a large amount of space it shouldn't!
nsumner
Novice
 
Posts: 5
Liked: never
Joined: Mon Feb 27, 2017 8:06 am
Full Name: Noach Sumner

Re: Primary ReFS Backup Repository Space Savings?

Veeam Logoby DaStivi » Tue Feb 28, 2017 9:48 am

nsumner wrote:What I am wondering is, is there ANY way that I can take advantage of block clone on old files? Any way at all! I'm out of space and want to use block clone to solve the problem but I must be able to use it for my archive!


i understand that like you have your vbks and copied them to ah new refs repo... you can not take advantage of refs fastclone instantly... you have to run a syn. full at least... that takes the fullamount of space for the vbk.... but after that fastclone on next syn full is used!

so you need at least 2times the vbk space.... than you can take advantage of refs!
DaStivi
Enthusiast
 
Posts: 50
Liked: 4 times
Joined: Tue Jun 30, 2015 9:13 am
Full Name: Stephan Lang

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Google [Bot], reportsrl and 67 guests