Comprehensive data protection for all workloads
stephen.loera
Service Provider
Posts: 39
Liked: 7 times
Joined: May 11, 2016 4:59 am
Full Name: Stephen Loera
Contact:

Re: GFS for primary backup jobs

Post by stephen.loera » 1 person likes this post

Hello,

Just wanted to add that I also would love to have GFS ability to be built into primary backups jobs.

Maybe this is a separate feature request, but I would like the option of having a, let's call it a "simple backup copy job." This type of job would literally just copy the backup files to another repository, like a file copy job. I have clients who want to send backup copies to a CCB Service Provider that exactly mirrors the source backup job.

Why? For example, a primary backup job can be configured to backup only on weekdays, but backup copy jobs are setups as "intervals". If a source or primary backup jobs only runs on weekdays, but the backup copy job runs everyday, backup copy jobs report errors on the weekend because there are no new restore points.

It would be nice to have a simple backup copy job that doesn't have all of the I/O overhead of a traditional backup copy job that operates in forever forward incremental mode.
DGrinev
Veteran
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: GFS for primary backup jobs

Post by DGrinev »

Hey Stephen,

Thank you for the feedback and feature request!

You should keep in mind that simple file copy can deliver a backup file to the target with the corrupted data due to numerous reasons (i.e. unstable connection).
Right now you are protected, since if the source is healthy the backup copy job can guaranty that the target backup will be healthy too, because it's working on the data block level.
Please review this thread for additional information.
evilaedmin
Expert
Posts: 176
Liked: 30 times
Joined: Jul 26, 2018 8:04 pm
Full Name: Eugene V
Contact:

Re: GFS for primary backup jobs

Post by evilaedmin »

Gostev wrote: Oct 24, 2018 8:37 pm So I wanted to ask everyone here: if it helps us to deliver this functionality sooner, would it be OK to only have GFS available for backup modes with periodic synthetic or active fulls - at least initially? Because then, all functionality comes down to NOT deleting some backup files during the retention policy processing.

This means it won't be available for forever forward incremental backup, like it is available with Backup Copy jobs (which are effectively forever forward incremental). Or for reversed incremental backup. There are significant extra complexities there, since these type of chains do not have "native" fulls - they need to be created. And this functionality is not easily portable to primary jobs.
Backup appliance user here: having GFS available for period synthetic fulls would be just about perfect and how I wish the product worked .
RGijsen
Expert
Posts: 124
Liked: 25 times
Joined: Oct 10, 2014 2:06 pm
Contact:

Re: GFS for primary backup jobs

Post by RGijsen »

+1 for us as well. We are a small company hosting RDS end Exchange services for a number of clients. We rent a rack in a datacenter, in which we have all our hardware, including a large Veeam repository machine. We have multiple gbit internet connections, but unfortunately a slow l2tp/ipsec site-to-site VPN which because of some latency on the remote-backup location I can't get past 70mbps.
We use want to use our remote-backup (copy jobs) to only store the last week of backups. We'd only need the copies if something went terribly wrong in the main datacenter. Because of the greatly bigger speeds and also much more available space in our primary repository, that's where I'd like to store the GFS backups, as restoring them from our remote machine if I needed to, it's just way, way too slow.

Apart from that, apart from Veeam I've never worked with a backup product so far that does NOT support GFS of some kind in the primary store.

So long story short, +1 for us too 😊
nwbc
Influencer
Posts: 20
Liked: 1 time
Joined: Feb 23, 2017 9:27 am
Full Name: nwbc

Re: GFS for primary backup jobs

Post by nwbc »

Hey there,
is there an update currently available on the progress of the GFS request?
DGrinev
Veteran
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: GFS for primary backup jobs

Post by DGrinev » 1 person likes this post

Thank you for keeping interest to the feature!
We cannot share any ETA for the upcoming features, but you can be sure that the work is in progress! Thanks!
Sean.Lucas
Service Provider
Posts: 8
Liked: 1 time
Joined: Jan 25, 2018 1:03 pm
Location: Ireland
Contact:

Re: GFS for primary backup jobs

Post by Sean.Lucas » 1 person likes this post

I also would like to see GFS retention on primary backup jobs.

In our scenario we would like to be able to use the primary backup job to directly backup to our DR site's repository instead of having to first create a local backup then a copy job out to our DR site.

We would also like this for Cloud Connect, having the option for our customers to backup directly to the Cloud Connect repository with GFS retention would be great!
deduplicat3d
Expert
Posts: 114
Liked: 12 times
Joined: Nov 04, 2011 8:21 pm
Full Name: Corey
Contact:

Re: GFS for primary backup jobs

Post by deduplicat3d »

+1 It would make my life a lot easier to have GFS on primary jobs. Backup copy would then be used just for emergencies where primary failed
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev » 1 person likes this post

evilaedmin wrote: Jan 07, 2019 4:00 amBackup appliance user here: having GFS available for period synthetic fulls would be just about perfect and how I wish the product worked.
Sounds good, thanks everyone for confirming. I am doing my best to make sure this implementation gets into the immediate update. Thanks!
Regnor
VeeaMVP
Posts: 934
Liked: 287 times
Joined: Jan 31, 2011 11:17 am
Full Name: Max
Contact:

Re: GFS for primary backup jobs

Post by Regnor »

Sean.Lucas wrote: Jan 29, 2019 11:05 am In our scenario we would like to be able to use the primary backup job to directly backup to our DR site's repository instead of having to first create a local backup then a copy job out to our DR site.
+1 We're already doing this for some customers as backup copy is just too slow for them.
dhayes16
Service Provider
Posts: 171
Liked: 19 times
Joined: Feb 12, 2019 2:31 pm
Full Name: Dave Hayes
Contact:

Re: GFS for primary backup jobs

Post by dhayes16 »

Yes. Add us to the list who would love this features. Veeam is awesome on so many levels but we went round and round logically on this thinking we were missing something. As an MSP the backup solution we have offered with storagecraft was exactly that. Several years of data onsite in gfs type format and only a few retention points on the cloud for DR. Veeam seems to be flipped around where they focus on offsite being gfs long term and local being short term. I am glad I am not the only one not understanding why it is not there.

For now we are doing primary backups with 7 day retention. Then a backup copy job based on that to a veeam cloud connect provider to give us the offsite insurance policy. Then we have a second backup copy job to local storage with the gfs functionality to give us the long term local retention. Obviously it is not efficient since we will have a duplicate vbk in there eating up space as well as wasted cpu.

So I am hoping this makes it in soon.

Thanks Veeam for listening to your customers.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev » 1 person likes this post

All, as we're working on this feature I wanted to ask your preference on GFS backup expiration settings. How would it be more convenient for you to specify the time period of how long those GFS restore points should be kept?

The current working idea is as follows:

Keep weekly full backup for X weeks
Keep monthly full backup for X months
Keep quarterly full backup for X years or quarters or months?
Keep yearly full backup for X years

Originally, I wanted to make it consistent everywhere (using the same period as the actual GFS type), however I realized that for me personally, counting the time period in quarters is somehow inconvenient. Is it just me?

An alternative I considered would be just to use days everywhere:

Keep weekly full backup for X days
Keep monthly full backup for X days
Keep quarterly full backup for X days
Keep yearly full backup for X days

But this will result in quite big numbers at the top end which are not immediately obvious what do they mean.

I guess my question is, how does your business define SLAs on how long the particular backup should be kept? It would be logical to match that, so that you don't have to do any conversions at all.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

Another application I have experience with solved this problem by including a dropdown to allow the user to specify the unit of measure for each retention point. One user could specify "3 months" and another could specify "90 days". I would set up mine in days because I would set the oldest backup to drop off a few days before the new backup came in.

For the current working idea, I agree that specifying quarterly retention in quarters is a bit awkward. "Months" makes more sense to me there. Though, I probably wouldn't use quarterly retention. I typically use weekly and monthly.
ChrisRoad
Service Provider
Posts: 22
Liked: 4 times
Joined: Oct 19, 2010 8:24 am
Full Name: Christoph Roethlisberger
Contact:

Re: GFS for primary backup jobs

Post by ChrisRoad »

@Gostev

I do like
Keep weekly full backup for X days
Keep monthly full backup for X days
Keep quarterly full backup for X days
Keep yearly full backup for X days
Yes, numbers will be big, but will remove the inherent uncertainness of what 3 weeks or 2 months really means. (i.e. does "two weeks" now mean that the backup is kept for 14 days and deleted on the 15th, or kept for 20 days and deleted on the 21st?)

Is "quarterly" still a requirement from customers?
We got rid of that long time ago and just use 6 or 9 monthly backups in cases we do somehow still have to cover the requirement of a traditional "quarterly" backup. (bust most often customer want monthly backups for up to a full year anyway)
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev » 1 person likes this post

@Marc more options is always an easy way out. However, more options => more code => more bugs => worse reliability, so we always strive to NOT provide options unless absolutely necessary. Flexibility to set the time period using both "90 days" and "3 months" ways does not seem essential at all (does not add extra value), so let's just make a call which way of the two is more convenient?

@Chris, good feedback that matches lots of things we were talking through with developers.

1. Yes, dropping quarterly completely is another option - less is better.

2. Yes, "2 months" can be anywhere betwween 59 and 62 days, depending on the month - so not deterministic. But in the retention logic, we can simply consider each month to be 31 days, to make sure backups are never deleted before any 2 months have passed (think about July + August scenario), so that it is not an issue. While deleting a backup 1 day later in some cases is obviously not a problem. This way, UI remains clean and simple (2 months), and backups are guaranteed to be stored for the specified period of time no matter what.

3. Days are great until you see something 183 days or 1825 days (hint, it means 6 months and 5 years). This will look really messy and unclear in UI for someone who did not set up the retention originally when creating the job, but has to manage one.

Answering your question, "keep backup for 2 weeks" means it will be deleted on 15th day since its creation (as soon as given backup age exceeds 2 weeks = 14 days). In other words, users will set the retention exactly according to the business requirements, without the need to translate "business" language into "technical". For example, for backups of a system maintaining the credit card payments log, they will set retention to 3 years - which is exactly what the corresponding law mandates.

BTW, in talking to some Veeam folks with backup administration IT background in parallel, so far the following UI is winning in their mind:

Keep weekly full backups for X weeks
Keep monthly full backups for X months
Keep yearly full backups for X years

Just about everyone agree that quarterly retention is dead.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

Quarterly retention is currently available in BCJs. The real trick will be able to figure out how to remove it from there in order to maintain consistency. :)
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev »

Right. We won't touch GFS in Backup Copy jobs for now at all (no time to make this change in the next release). We'll deal with it later, but end state is BCJ retention becoming the same as what we're discussing here, of course.
dhayes16
Service Provider
Posts: 171
Liked: 19 times
Joined: Feb 12, 2019 2:31 pm
Full Name: Dave Hayes
Contact:

Re: GFS for primary backup jobs

Post by dhayes16 » 1 person likes this post

Personally I like..

Keep weekly full backups for X weeks
Keep monthly full backups for X months
Keep yearly full backups for X years

I really do not think we sacrifice anything by keeping it simple here. This is really how our SLA for on prem would look.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev »

Thanks Dave!
GreenAlpha55
Enthusiast
Posts: 32
Liked: 3 times
Joined: Oct 25, 2018 2:20 pm
Contact:

Re: GFS for primary backup jobs

Post by GreenAlpha55 »

I would like to vote for GFS on backup jobs as well. Because I can't I am forced to have a backup copy job (and use double the storage for no reason).
engedib
Influencer
Posts: 11
Liked: 7 times
Joined: Jan 04, 2019 2:40 am
Contact:

Re: GFS for primary backup jobs

Post by engedib » 2 people like this post

dhayes16 wrote: Feb 20, 2019 5:09 am Personally I like..

Keep weekly full backups for X weeks
Keep monthly full backups for X months
Keep yearly full backups for X years

I really do not think we sacrifice anything by keeping it simple here. This is really how our SLA for on prem would look.
This is exactly what we need as well :) Easy, simple.
DerOest
Enthusiast
Posts: 71
Liked: 42 times
Joined: Oct 30, 2015 10:10 am
Contact:

Re: GFS for primary backup jobs

Post by DerOest »

dhayes16 wrote: Feb 20, 2019 5:09 am Personally I like..

Keep weekly full backups for X weeks
Keep monthly full backups for X months
Keep yearly full backups for X years

I really do not think we sacrifice anything by keeping it simple here. This is really how our SLA for on prem would look.
GreenAlpha55 wrote: Feb 21, 2019 9:18 pm I would like to vote for GFS on backup jobs as well. Because I can't I am forced to have a backup copy job (and use double the storage for no reason).
+1

This will make configuration a lot easier (less jobs to take care of etc.) and save the additional copy space (which is either wasted, and/or puts more load on the storage system running deduplication etc.)

Btw, our take on "3-2-1" is like:
a) Primary Data (2 Datacenters, redundant/mirrored storage)
b) Backup to onPrem Storage at a third location (multiple 10Gb lines available, with the whole GFS scheme)
c) Backup to Tape as Desaster Recovery/Offline/Airgap (so not keeping a lot of tape around, just the last 4 Weekly & 2 Monthly for DR case)
GreenAlpha55
Enthusiast
Posts: 32
Liked: 3 times
Joined: Oct 25, 2018 2:20 pm
Contact:

Re: GFS for primary backup jobs

Post by GreenAlpha55 » 1 person likes this post

I have a hard time understanding "3-2-1" with Veeam. Veeam essentially forces "4-2-1". If you're not doing "4-2-1", then if your offsite data is destroyed your GFS backups are gone.

"4-2-1" meaning;
Production > Backup Job > Backup Copy w/GFS > Backup Copy w/GFS (offsite) = necessary for GFS at 2 locations.

"3-2-1" currently;
Production > Backup Job > Backup Copy w/GFS (offsite) = GFS only offsite. If offsite goes down, GFS backups are gone.

"3-2-1" would only be possible when GFS is implemented on backup jobs. = GFS at 2 locations. Eliminates need for local backup copy. YES!
Production > Backup Job w/GFS > Backup Copy w/GFS (offsite)
KelKnowles
Service Provider
Posts: 30
Liked: 2 times
Joined: Sep 15, 2012 8:01 pm
Full Name: Kelly Michael Knowles
Contact:

Re: GFS for primary backup jobs

Post by KelKnowles »

I constantly run into a need for this feature. One customer I have hourly backups on disk with daily synthetic fulls but they also want long term past 2 weeks or so on the same primary storage. I already adhere to the 3-2-1 rule with cloud and tape copies so having to do a local backup copy uses up additional space and also disk IO that could just be a fast clone / spaceless full.
Kelly Knowles
Principal Systems Architect at PNJ Technology Partners
Veeam Certified Architect and Veeam Certified Engineer - Advanced: Design & Optimization
petben
Novice
Posts: 9
Liked: never
Joined: Mar 18, 2019 9:50 am
Full Name: Pete Benn
Contact:

[MERGED] Multiple Schedules

Post by petben »

I see this has been discussed since 2011, when are Veeam going to allow multiple schedules to be set on the same job. so for example a user can set a daily/weekly/monthly backup schedule. this seems like a basic, core part of backup.

Yes you can setup multiple jobs but there's no deduplication across jobs.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS for primary backup jobs

Post by foggy » 1 person likes this post

Hi Pete, please see discussion above for the answer. Thanks!
arkroyal
Lurker
Posts: 2
Liked: never
Joined: Apr 10, 2015 4:12 pm
Contact:

Re: GFS for primary backup jobs

Post by arkroyal »

Love VBR for the most part, but the backup and backup copy retention options are not exactly the most intuitive to try and figure out. I have got it to the point where I am (reasonably) happy with my retention policies, but having the backup copy job running for no purpose, i.e. no new backup to copy, seems to me to be a waste of resource. Another vote here for:

Keep weekly full backups for X weeks
Keep monthly full backups for X months
Keep yearly full backups for X years
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev » 2 people like this post

Yes, this ended up to be the final design for this feature.
mjm599
Lurker
Posts: 2
Liked: never
Joined: May 28, 2019 3:05 pm
Full Name: mjm599
Contact:

Re: GFS for primary backup jobs

Post by mjm599 »

Hi,

Does anyone know when this functionality will be available? Or any advancement on the current configuration options available for Azure object storage integration?

Thanks,
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS for primary backup jobs

Post by veremin » 1 person likes this post

We're working on this feature, but cannot provide ETA at the moment. Thanks!
Post Reply

Who is online

Users browsing this forum: Semrush [Bot] and 348 guests