GFS for primary backup jobs

Availability for the Always-On Enterprise

Re: GFS for primary backup jobs

Veeam Logoby jmmarton » Thu Jan 19, 2017 2:20 pm

In your case, you could theoretically use ReFS still and leverage the new block clone feature in ReFS 3.1. If you leverage that 50 TB disk array and make your primary repo Server 2016 formatting the disk on that array as ReFS, then creating a second folder for a second repo in order to use a BCJ with GFS would still have the ReFS advantages for all the GFS restore points. After all, each of those is a synthetic full. If you've already formatted your primary repo as NTFS or if it's not even Server 2016, then you'd have to create an additional LUN which is attached to a server running 2016 and then use ReFS.

Joe
jmmarton
Veeam Software
 
Posts: 719
Liked: 85 times
Joined: Tue Nov 17, 2015 2:38 am
Location: Chicago, IL
Full Name: Joe Marton

Re: GFS for primary backup jobs

Veeam Logoby EricJ » Thu Jan 19, 2017 2:29 pm

Thanks for the reply, Joe. We do have the 50TB array formatted as ReFS 3.1, and it works amazingly well for the primary backups. Synthetic fulls are created very fast and take up very little space. However, it seems when I do my first backup copy to a different folder on the same volume, it creates a full copy of all of the data. I have a 5TB job, and if I do a backup copy of that job, it seems to make a bit-for-bit copy and I lose 5TB of free space on disk. I haven't tested yet, but my hunch if that subsequent backup copies will leverage that initial backup copy and take up little space (since the 9.5 documentation says fast clone works on backup copies), but it still requires me to use up 10TB of space for the one 5TB job. It seems since I'm "tricking" Veeam into thinking my second repository is on a different disk, it doesn't bother to attempt to use ReFS to clone the primary copy into a backup copy. Or maybe I have something misconfigured? I will know more after a few more backup copies run.

Thanks,
Eric
EricJ
Influencer
 
Posts: 11
Liked: 1 time
Joined: Thu Jan 12, 2017 7:06 pm

Re: GFS for primary backup jobs

Veeam Logoby jmmarton » Thu Jan 19, 2017 4:06 pm

You are spot on. The BCJ will initially create an additional copy of all the data (which in some regards isn't necessarily a bad thing), but any GFS points created will be synthetic fulls that should fully leverage block clone within ReFS. So there's some additional space used but in the end you'll still be getting the benefits of ReFS, and with the 19 GFS retention points (4 weeklies + 12 monthlies + 3 yearlies) you'll still have immense space savings.

Joe
jmmarton
Veeam Software
 
Posts: 719
Liked: 85 times
Joined: Tue Nov 17, 2015 2:38 am
Location: Chicago, IL
Full Name: Joe Marton

Re: GFS for primary backup jobs

Veeam Logoby mkaec » Thu Jan 19, 2017 5:28 pm

It makes sense that ReFS 3.1 goodness would not apply to backup copies. One option is to ditch ReFS and use NTFS. Then you could turn on NTFS deduplication which would dedup the backup copies against the primary jobs. But, I don't know if I would consider that a "better" solution. There would be a lot lost in moving away from ReFS.

I think the ReFS 3.1 cloning functionality actually adds to the case to have GFS available to primary jobs. Why force someone to use double the repository space just to keep long-term backups at the primary site?

Keeping long term backups off-site, when both the primary and secondary repositories are disk based, actually seems counter-intuitive to me. My use for long term backups is to handle the restore request of "I accidentally deleted a file 2 months ago and am just now getting around to asking about it." If I need to go to the off-site backup for DR, I'm going to want to restore from a recent backup.
mkaec
Expert
 
Posts: 172
Liked: 43 times
Joined: Thu Jul 16, 2015 1:31 pm
Full Name: Marc K

Re: GFS for primary backup jobs

Veeam Logoby jmmarton » Thu Jan 19, 2017 5:50 pm

I would think the opposite: ReFS builds the case of having GFS in copy jobs. The reason being is that if the GFS restore points were in the backup job, all of those are pointers to the same set of blocks as the regular backup. If you run into a problem all those GFS restore points are worthless. But if you have an additional copy of all those blocks, which BCJ requires, then you're less likely to lose all your GFS restore points.

This doesn't remove the overall use case of having GFS in primary backup jobs. I can see reasons for it and understand the feature request. I just don't think ReFS is a reason. :-)

Joe
jmmarton
Veeam Software
 
Posts: 719
Liked: 85 times
Joined: Tue Nov 17, 2015 2:38 am
Location: Chicago, IL
Full Name: Joe Marton

Re: GFS for primary backup jobs

Veeam Logoby EricJ » Thu Jan 19, 2017 9:43 pm

I do agree that having separate copies of the data is better, and once we get a second disk array to replace tapes at our third location, I won't care as much about this issue.

However, I do think ReFS makes a stronger case for GFS in primary backups. In the past, it may have been too slow or would consume too much disk space to keep 30 daily, 4 weekly, 12 monthly, and 3 yearly at the primary site. So, we'd keep 30 dailies on primary disk, and the GFS at a secondary site on tape.

With ReFS, keeping the GFS copies on site is very fast and uses very little space. So now it's given me the desire to have GFS at the primary and the backup site. This way, there's nothing lost if a single backup location goes away. Plus, our primary backup repo is much faster than our secondary (tape) repo, so for quick restores I'd love to have them also available on disk. Maybe you mean that this was already possible somehow before ReFS, but for me that was the trigger that made me wish for it.

I understand that you want to encourage customers to keep at least two copies of backup data, and so it's not currently allowed. But it would be nice if it could at least be enabled through a registry key after some kind of warning or disclaimer. 8)
EricJ
Influencer
 
Posts: 11
Liked: 1 time
Joined: Thu Jan 12, 2017 7:06 pm

Re: GFS for primary backup jobs

Veeam Logoby jimmycartrette » Thu Feb 02, 2017 2:24 pm

I agree - just coming from Netbackup, I'm finding setting GFS style retention for things difficult. In Netbackup, I'd have something like 30 days of daily, and weekly for a year on my primary, all copied to my DR repository. Now that I've already created a primary site repo with my free SAN space with 2016 ReFS, I'm finding it annoying trying to get the right retention and keeping the ReFS benefits of space savings.
jimmycartrette
Novice
 
Posts: 9
Liked: 2 times
Joined: Thu Feb 02, 2017 2:13 pm
Full Name: JC

[MERGED] Question about archive

Veeam Logoby joshgfeller » Thu Mar 30, 2017 2:13 pm

Hello, I know you can create archive points from backup copies for monthly, quarterly, and yearly. However, if I want to keep some of these longer range archival points on my main repository that is also running my regular backup jobs can you set that somewhere in the backup job instead of a backup copy job? If I try to setup a backup copy job with archival and point it to the same repository it won't allow it.
joshgfeller
Influencer
 
Posts: 10
Liked: never
Joined: Fri Apr 05, 2013 5:14 pm
Full Name: JOsh Gfeller

Re: Question about archive

Veeam Logoby csinetops » Thu Mar 30, 2017 2:21 pm

Nope. You have to backup copy them to an alternate repository. This is by design, you don't really want long term retention points on your main repository. You can manually or script to copy the files to alternate/removable media for long term retention too. This is what I used to do before getting my AltaVault appliance for long term retention.
csinetops
Expert
 
Posts: 101
Liked: 15 times
Joined: Fri Jun 06, 2014 2:45 pm
Full Name: csinetops

Re: GFS for primary backup jobs

Veeam Logoby v.Eremin » Thu Mar 30, 2017 6:37 pm

However, if I want to keep some of these longer range archival points on my main repository that is also running my regular backup jobs can you set that somewhere in the backup job instead of a backup copy job?

We kind of try to impose the best practices, according to which storing two copies of the same data within the same location is not a good thing. But, as mentioned above, if you feel safe, create additional folder on the same device, assign repository role to it and point backup copy job to it. Thanks.
v.Eremin
Veeam Software
 
Posts: 12694
Liked: 920 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

[MERGED] Any plan to make GFS retention available on regular

Veeam Logoby tntteam » Tue Apr 04, 2017 2:32 pm

Hi there,

Is there any plan to let us have a GFS like retention in regular backup chains ?

ATM we are using backup copy jobs, but it's just a complete waste of time and resources since our backups and backups copy are on the same storage array.

Example :
We have like 1 month rollback requirement with 1 point / week, but only 7 days rollback requirement for daily backups. We would love being able to have 7 restore points + 4 weekly restore points, without having to waste time copying files.

I forgot to mention that 90% of problems we got with veeam are from backup copy jobs (retention = R = never deleted file, or vib files sometimes present whereas only full is selected, loss of sync = need to recreate backup copy jobs, etc... Anyone using backup copy jobs know what I mean).

Is there any plan to include GFS retention in regular backup chains ?

Where can I submit this feature request officially as a customer that pay its licences ?


Thanks
tntteam
Enthusiast
 
Posts: 44
Liked: 3 times
Joined: Fri Aug 28, 2015 12:40 pm
Full Name: tntteam

Re: GFS for primary backup jobs

Veeam Logoby foggy » Wed Apr 05, 2017 3:27 pm

All feature requests submitted through the forums are "officially" accepted by the product management team, so thank you for this feedback.
foggy
Veeam Software
 
Posts: 14310
Liked: 1052 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: GFS for primary backup jobs

Veeam Logoby jimmycartrette » Thu Apr 06, 2017 12:25 pm

I'm curious about the ReFS users here who have done a setup where they have the backup copy GFS set up on the primary and secondary sites. What does that look like?
On your primary site, do you have something like E:\backups and E:\GFS
Then you have your normal backup jobs - what's typical for your backup jobs? DO you do a small number (or one) number of retention points and handle everything else as BC? Or do you primarily use backup jobs for primary and then only bring in BC GFS for jobs that require it on primary?
I'm about to try to structure my infrastructure. Ideally mirror everything on primary and secondary. ReFS on primary and secondary. Primary storage has Dedupe + Compression on the array, secondary has Compression on the array.

Should I create E:\backups, do retention of maybe 1 or 2 points for 'staging', and then use backup copy jobs to replicate this to the daily and GFS rentention I want to E:\GFS (and secondary site S:\GFS)?

I'm just curious how other people are structuring this thing before I start designing policies. I'm lucky to have dedup inline on my storage for primary so a 'staging' area won't lose me that much space.

Thanks
jimmycartrette
Novice
 
Posts: 9
Liked: 2 times
Joined: Thu Feb 02, 2017 2:13 pm
Full Name: JC

Re: GFS for primary backup jobs

Veeam Logoby EricJ » Thu Apr 06, 2017 8:54 pm

Jimmy -

We have our primary backups on E: and our GFS on F: - only because we had reached our array's max volume size for E:. Otherwise yes, just two different folders on the same drive would be essentially the same thing.

We are still tweaking things to fit into our lack of disk space until summer when we hope to expand, but currently we keep 30 restore points (30 days) on E:. Then we backup copy everything to F:, where we keep 2 daily restore points (the minimum allowed), and retain 12 monthly backups as well. The bummer, of course, is that we have to duplicate our entire environment on F: just to retain the monthly backups we need. What's frustrating is that this isn't a technical hurdle, or an ReFS shortcoming - it's just Veeam's code not allowing GFS retention alongside primary backups. A registry key to override this would be nice. We don't need this safeguard since our primary backups are already off-site, and again to tape at a third site. I'd love to efficiently retain a big timespan of backups on disk for quick and easy restores. No worries about all our eggs in one disk array, since all the same data is off to tape elsewhere.

But yes - you are lucky to have inline dedupe since you won't be nearly as burdened by having a second full copy of your data on the same array. :D
EricJ
Influencer
 
Posts: 11
Liked: 1 time
Joined: Thu Jan 12, 2017 7:06 pm

Re: GFS for primary backup jobs

Veeam Logoby jimmycartrette » Fri Apr 07, 2017 11:56 am

Thanks Eric, that helps.

Since I only need GFS on select servers, I'll set up something similar - using standard backup to (PRIMARY)R:\Backup, using backup copy to (DR)R:\Backup without GFS for most jobs, and for jobs needing GFS, a standard backup with needed daily rentenion, plus a GFS job with two required retentions plus GFS rules to (PRIMARY)R:\GFS and a separate backup copy job with the daily retention setting from the backup job plus the GFS settings going to (DR)R:\GFS.

Server 1 (2 weeks daily only)
Backup to (Production)R:\Backup
Backup Copy (same 14 points) to (DR)R:\Backup

Server 2 (2 weeks daily, Monthly for a Year, Yearly for 7 years)
Backup to (PRODUCTION)R:\Backup, 14 points
Backup copy to (PRODUCTION)R:\GFS, 2 points, monthly for year, yearly for 7
Backup copy to (DR)R:\GFS, 14 points, monthly for year, yearly for 7

A bit sloppy but I think I can make it work, based on my thinking (although I'm not as familiar with the system as you already using it). It should take care of ReFS waste on (PRODUCTION) with inline dedup, and the setup should cause the ReFS chains to fully map together in my (DR) site which does not have incline dedup, only compression.
jimmycartrette
Novice
 
Posts: 9
Liked: 2 times
Joined: Thu Feb 02, 2017 2:13 pm
Full Name: JC

Previous

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Google Feedfetcher and 26 guests