GFS for primary backup jobs

Availability for the Always-On Enterprise

Re: Feature Request and Review

Veeam Logoby VladV » Sun Aug 23, 2015 9:39 am

Do you make copies of your backups?


Sure, we have onsite a primary storage location for the Backup Job and a robocopy job at the end of the week on a different storage medium. We also have a Backup Copy job for DR purposes on an offsite location storage. Not to mention the replication part.

Being able to smartly manage (with GFS) the onsite location backup is helpful first for being able to cram in more restore points (with or without dedupe) and second, for having those restore points close and with fast restore speeds compared to offsite backups.

We consider that it's not a good option to have a Backup Copy job do the GFS part on the same volume. It creates an unnecessary overhead in management and resources to reprocess restore points and extract VBKs and VIBs at each cycle. For a simple Backup Job (forward incremental), automatically deleting (according to a GFS policy) the increments and keeping the VBKs (eg: 1 per previous years, 12 in current year and 4 in the current month), creates more space with the added benefit of, like I said above, having a properly managed chain close to the restore location.

A thing to mention is that Backup Copy Jobs with the Forever Forward Incremental (I believe that is what it's called) schematic is not very good with dedupe. Constantly modifying the VBK decreases the dedupe ratio. I'll give you an example taking our two backup repositories (the onsite - simple backup job and offsite - backup copy job):

- the backup job has a restore point number set to 60 - daily and an active full on weekends
- the backup copy job has a restore point number set to 60 - every 3 days

- Onsite dedup performance: 30TB savings with 78% dedup rate
- Offite dedup performance: 4,4TB savings with 49% dedup rate

I understand that the basic backup job is not considered to be a historical backup solution but, maybe it should be. Not at the same level of complexity as the Backup Copy Job but much simpler like I mentioned above.
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Feature Request and Review

Veeam Logoby Shestakov » Sun Aug 23, 2015 8:36 pm

Vlad, thanks for the detailed feedback!
VladV wrote:Being able to smartly manage (with GFS) the onsite location backup is helpful first for being able to cram in more restore points (with or without dedupe) and second, for having those restore points close and with fast restore speeds compared to offsite backups.

Agreed. Also you are right, that pointing backup copy job to the same repository doesn`t make a lot of sense.
However, it seems more logical to keep short-term backups onsite on the faster repository and historical ones offsite on the more reliable repository.
VladV wrote:A thing to mention is that Backup Copy Jobs with the Forever Forward Incremental (I believe that is what it's called) schematic is not very good with dedupe. Constantly modifying the VBK decreases the dedupe ratio.

We are planning to provide an option of active full for backup copy jobs in the upcoming version, so that will not be an issue.

Anyways, your request is counted. Thanks!
Shestakov
Veeam Software
 
Posts: 4757
Liked: 385 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

[MERGED] *Feature Request*

Veeam Logoby SyNtAxx » Sun Feb 07, 2016 8:02 pm

GFS support on 'regular' cycle backups.

-Nick
SyNtAxx
Expert
 
Posts: 127
Liked: 13 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: GFS for primary backup jobs

Veeam Logoby SyNtAxx » Sun Feb 07, 2016 9:44 pm

So, after reading the thread, I don't understand why we don't have GFS on the standard backups? There are many reasons that it could be useful. Is Veeam trying to protect us from ourselves I guess?
SyNtAxx
Expert
 
Posts: 127
Liked: 13 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: GFS for primary backup jobs

Veeam Logoby Gostev » Mon Feb 08, 2016 2:09 am

Main reason is that GFS on primary backup repository goes against our reference architecture. And yes, at the same time this also helps to protect inexperienced backup admins from sticking with a single copy of backups.
Gostev
Veeam Software
 
Posts: 21354
Liked: 2333 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: GFS for primary backup jobs

Veeam Logoby SyNtAxx » Mon Feb 08, 2016 3:20 am 1 person likes this post

Why cant I do what is best for my data and make my own decisions? I feel not having GFS on regular backups Is really inflexible and forces me to shuttle data all over my data center in order to maintain proper retention periods which is really in efficient, time consuming, a waste of network bandwidth and so on. My company has a significant investment in Veeam (Enterprise Plus, over 100 sockets),sure we may not be the largest install out there, but forcing customers to your ideals is silly in my opinion. We all know we should have multiple copies of data, but let me achieve that in a manner that fits my needs. Right now all I can cover is basic 30 retention with out having to shuffle HUNDREDS of TB around hoping to get the retention I need. Your product has a lot of great features over other platforms I've used int he past. We left HP Data Protector because it was clunky when it came to protecting VMs, but I'll take that and sleep safe knowing I can easily set retention on any backup at any point in time to any length of time I desire at a moments notice.
SyNtAxx
Expert
 
Posts: 127
Liked: 13 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: GFS for primary backup jobs

Veeam Logoby Gostev » Mon Feb 08, 2016 8:25 pm

SyNtAxx wrote:We all know we should have multiple copies of data, but let me achieve that in a manner that fits my needs. Right now all I can cover is basic 30 retention with out having to shuffle HUNDREDS of TB around hoping to get the retention I need.

Now I am curious how can you achieve multiple copies of data without physically copying it, can you share? I assume you used GFS for primary with HP Data Protector - what was your way of creating an additional copy of backups back then? Thanks.
Gostev
Veeam Software
 
Posts: 21354
Liked: 2333 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: GFS for primary backup jobs

Veeam Logoby SyNtAxx » Mon Feb 08, 2016 10:24 pm

Simply put, we didn't. But that is the risk *we* decided/needed to take given our situation then. We also didn't have the same number of machines to protect or the same gear we currently enjoy. You're missing the point. I should be able to make copies of my data, or not, in the manner I see fit. Maybe we don't have the storage required to keep 3 copies of data around. The way it is now, I cant even keep one full copy (The original) of data for 2 years. I cant go in and manually set my data to 'never expire' based on a current and breaking legal hold ( like i could in DP, etc).

In my opinion that is inflexible. In addition, I opened a case today for GFS inquires. There is no exception logic on your GFS retention. The example I presented above (a few failed vms, manually, but post GFS trigger time/date) does not make it into the GFS retention policy. The solution proposed by Veeam is to go in and manually *change* the Month End date (or what ever applies to you) on the failed copy job(s) to a date that will trigger a new Month end job copy. The issue there is, firstly, that is a lot of administrative over head (to change and revert for next month x your job count). Secondly, it creates additional redundant copies on storage which I may not have space to hold an entire job, potentially terrabytes. I'm sorry, I don't find that to be an enterprise level solution. There needs to be more flexibility on retention and how and where it is applied. I am open to suggestion, of course and willing to provide any additional feedback that might be required to create a solution.
SyNtAxx
Expert
 
Posts: 127
Liked: 13 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: GFS for primary backup jobs

Veeam Logoby TroyResources » Tue Feb 09, 2016 6:29 am

TroyResources
Enthusiast
 
Posts: 34
Liked: 6 times
Joined: Tue May 26, 2015 7:44 am
Full Name: TroyResourcesLimited

[MERGED] Suggestion for a different way of handling backups/

Veeam Logoby werten » Tue Jun 07, 2016 7:17 am

I think it's a real shame: I really believe (and others as well, apparently, with convincing arguments) that this would be very beneficial for many users in different scenarios. It would add flexibility and intuitiveness to an already great and reliable backup system. I realize that it is sometimes difficult to step away from how things have been working for a long time, but since the basic idea is already implemented in backup copy and would not hinder or prohibit current backup schemes, why not take the additional step and add it to the primary backup jobs as well? At least consider this. I believe many, certainly new and novice users, would greatly appreciate it, as it would make creating backup tasks much easier and understandable, and less backup jobs would be needed in many cases.
werten
Influencer
 
Posts: 15
Liked: never
Joined: Thu Apr 14, 2016 3:16 pm
Full Name: werten

Re: Suggestion for a different way of handling backups/versi

Veeam Logoby werten » Tue Jun 07, 2016 7:29 am

The weekly, monthly and yearly retention settings (as they are implemented in the backup copy job) could simply be added to the Storage page of the Edit Backup Job dialog, or to the Advanced setting lying one level below that page...
werten
Influencer
 
Posts: 15
Liked: never
Joined: Thu Apr 14, 2016 3:16 pm
Full Name: werten

Re: Suggestion for a different way of handling backups/versi

Veeam Logoby jazzoberoi » Tue Jun 07, 2016 1:26 pm

+1 to adding the GFS retention settings to the primary backup job as well. This way, the backup-copy job would be used only to "COPY" the backup files to a secondary location as its name implies. Having a primary backup job and then creating a secondary Backup-Copy job only to avail of the GFS retention seems unnecessary in my opinion.
jazzoberoi
Enthusiast
 
Posts: 63
Liked: 13 times
Joined: Wed Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi

Re: Suggestion for a different way of handling backups/versi

Veeam Logoby foggy » Tue Jun 07, 2016 1:40 pm

We appreciate your feedback. The main considerations behind the current implementation were explained here.
foggy
Veeam Software
 
Posts: 14560
Liked: 1060 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: GFS for primary backup jobs

Veeam Logoby mkaec » Mon Jun 13, 2016 2:08 pm 1 person likes this post

+1 for adding GFS retention to regular backup jobs. We work around this by having the primary repository and the copy repository on the same volume. Dedup minimmizes the storage impact of the copies, but the setup is fraught with inefficiencies. As far as 3-2-1, the appliance replicates the backup files to an off-site appliance. So, we are good there. We could save wasted time and compute, and reduce complexity, if GFS were available in standard backup jobs.
mkaec
Expert
 
Posts: 174
Liked: 45 times
Joined: Thu Jul 16, 2015 1:31 pm
Full Name: Marc K

Re: GFS for primary backup jobs

Veeam Logoby ginux » Wed Jun 22, 2016 1:05 pm 1 person likes this post

+1 for GFS retention to regular backup jobs... it should be a great feature added and I agree with all the above considerations. I hope It will release as soon as possible (v.10?); I'm going crazy to move 20 TB data for monthly and yearly vaulting.
ginux
Novice
 
Posts: 3
Liked: 1 time
Joined: Mon Jul 30, 2012 10:05 am
Full Name: Gino Calzavara

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: No registered users and 23 guests