Comprehensive data protection for all workloads
dguidry
Novice
Posts: 4
Liked: never
Joined: Sep 30, 2016 4:57 pm
Contact:

Re: GFS for primary backup jobs

Post by dguidry »

Now that I've switched to ReFS for my primary backup target, I would absolutely love to be able to fully leverage ReFS block cloning to keep historical backups at our primary site. We already have offsite backups as one should. However, It would be convenient to have a GFS chain at our primary site, if nothing else for the occasional email or file restore from 6 months ago that a user might request. The problem I'm running into is that, in order to keep GFS backups on at our primary site, I have to create a second repo on that server, set up a backup copy job and then run that. Because block cloning can't be leveraged across jobs, that means the base disk space requirements are doubled for us. In other words, let's say an active full of all our VMs would take up 10TB, well that doubles to 20TB in order to use backup copy jobs that can then take advantage of block cloning for longer term GFS purposes. Suddenly the whole notion of "spaceless GFS backups" becomes decidedly less spaceless, at least initially. This is a perfect scenario where having either GFS capability within a normal job, or the ability to link backup chains between a normal and copy job in such a manner where the copy job could leverage block cloning based on the primary job would be ideal. We're a pretty small shop and are looking for ways to be more efficient with storage wherever we can.

I realize this might be abused, but it could be somewhat mitigated with some friendly warnings, or something like that. Again, this has never really been something that mattered to me, but now that I'm utilizing ReFS, I'm realizing how much space I could be saving on the base disk requirements of a backup copy job.

Thanks!
Daniel
Brigham
Lurker
Posts: 2
Liked: 1 time
Joined: Sep 28, 2016 4:39 pm
Full Name: Brigham Mirabelli
Contact:

[MERGED] Feature Request: Extended Retention in Primray Back

Post by Brigham »

Feature Request:
Please consider adding extended retention to the primary backup job. This is useful for clients that want to keep years of data at the primary site but only want a small amount data in the copy job offsite.

The only way to do this now is to do a copy job of the backup to the same repo but that is a waste of resources.

Thanks,
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS for primary backup jobs

Post by foggy »

Hi Brigham, thanks for the request, your post has been merged into a discussion regarding similar matter.
tturnipseed
Lurker
Posts: 2
Liked: 1 time
Joined: Jun 06, 2018 6:39 pm
Full Name: Tate Turnipseed
Contact:

Re: GFS for primary backup jobs

Post by tturnipseed »

I would also like to cast my vote for adding extended retention options to the primary backup job. I do not see this extended retention as a replacement for an additional copy in a different location, but with the space savings of fast clone, it seems like a useful data set to have in multiple places.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

The way it is now I can create a primary job with short retention and then use GFS in a copy job for long retention. If my environment suffers a disaster and I need to recover from DR, I'm going to want to restore from the most recent backups. So, having long term backups there doesn't help me much. It's when a user finds out he made a critical mistake in a file 6 weeks ago that I'll need to go to the long term backups. The current paradigm has me dragging that file over the WAN from the DR site when I'd prefer to be retrieving it out of the local backups.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

I guess the counter argument to my own argument is if you wanted to use very fast disks for the primary repository in order to provide the best possible performance for restore and instant recovery. In that case, you would want your historical backups elsewhere on less expensive storage.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS for primary backup jobs

Post by veremin »

The current paradigm has me dragging that file over the WAN from the DR site when I'd prefer to be retrieving it out of the local backups.
If you want, you can create a local copy job and make it create GFS restore points locally. Just create a separate folder, assign repository role to it and leverage it then as a target repository for the backup copy job. Thanks.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

That is a work-around. But, it's not ideal as there is quite a bit of extra disk space used.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS for primary backup jobs

Post by veremin »

Even if we had GFS restore point for primary backup job, those points would occupy the very same amount of space. So, I'm not sure whether I'm following you on that. Thanks.
Dsgrievew
Lurker
Posts: 1
Liked: never
Joined: Aug 01, 2017 8:29 pm
Full Name: William Grieve
Contact:

Re: GFS for primary backup jobs

Post by Dsgrievew »

I definitely agree that the GFS functionality should be independent of the Backup Copy functionality. Having worked with numerous other backup technologies, properly implementing both GFS schemes and backup copies schemes in Veeam is one of the hardest things to figure. I would like lot more control of how weeklies, monthlies and yearlies are created and what backup files are copied. Normally it is the long term retention backups you want multiple copies of. I've found Veeam's scheduling to be both really flexible for some things but really limiting for other things.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

v.Eremin wrote:Even if we had GFS restore point for primary backup job, those points would occupy the very same amount of space. So, I'm not sure whether I'm following you on that. Thanks.
With the current capabilities, if I want GFS on the primary repository, I've got to create a BCJ to save into a different folder in the repository. So, at least some of the daily backup VBK and VIBs are duplicated between the main job and the GFS job.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: GFS for primary backup jobs

Post by veremin »

It holds true only until backup job retention policy removes those intersecting points, and the additional amount of space temporarily occupied by such restore points should be negligible. Thanks.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

I would think it is at least one additional VBK for each job. That could add up in large deployments. Plus, there is the time and resources spent making all those extra copies.
KelKnowles
Service Provider
Posts: 30
Liked: 2 times
Joined: Sep 15, 2012 8:01 pm
Full Name: Kelly Michael Knowles
Contact:

Re: GFS for primary backup jobs

Post by KelKnowles »

I think it would be much better if REFS would be able to be fully leveraged on the primary backup job to create the GFS copies. Of course you want to try to adhere to 3+2+1 rule however this just makes it so you end up with 4+2+1 if you have a large backup repository and want to keep your historical backups onsite. Creating a seperate folder for backup copies doesn't leverage REFS technology.

Right now I am forced to do a backup copy to a separate folder for a 30TiB job and it would be really nice to be able to leverage REFS for this not just in the space savings but the time required.

Wouldn't it also be great if we could also create a weekly monthly or yearly on demand from an existing .vbk and have it use REFS to become a spaceless full copy?
Kelly Knowles
Principal Systems Architect at PNJ Technology Partners
Veeam Certified Architect and Veeam Certified Engineer - Advanced: Design & Optimization
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: GFS for primary backup jobs

Post by Shestakov »

Hello KelKnowles,
Having 2 copies on ReFS it's same as 1 copy in terms of data safety. If the data blocks are corrupted or disk is broken, both copies will be lost.
However, we will count +1 to primary job GFS requests.
Thanks!
MarcinS
Enthusiast
Posts: 29
Liked: 2 times
Joined: May 18, 2015 8:31 am
Contact:

Re: GFS for primary backup jobs

Post by MarcinS »

please count 1 from me for adding GFS just in backup job
nwbc
Influencer
Posts: 20
Liked: 1 time
Joined: Feb 23, 2017 9:27 am
Full Name: nwbc

Re: GFS for primary backup jobs

Post by nwbc »

+1 from me
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec » 1 person likes this post

Are we almost there? How many more +1's do we need to get this onto the schedule?
nwbc
Influencer
Posts: 20
Liked: 1 time
Joined: Feb 23, 2017 9:27 am
Full Name: nwbc

Re: GFS for primary backup jobs

Post by nwbc »

As I've researched in the forums about this. There must be some kind of technical complications as stated at least once somewhere a couple months/years? ago. Veeam is quite powerful and works well, also uses resources in a good manner making it viable for small business customers to stay on 1Gb Infrastructure longer - as is the case with us. However, it'd be impossible for me to make GFS work at our current infrastructure if I'd have to move around town with my data like that.

It's obviously not a question about want nor about it being reasonable. It's about time/investment or complications. Undoubtfully, it is "possible", and expected by the customers to provide that feature.
kbe4
Novice
Posts: 3
Liked: never
Joined: Mar 23, 2018 11:02 am
Full Name: Kimmo Berghäll
Contact:

Re: GFS for primary backup jobs

Post by kbe4 »

Upvote.
raudi
Novice
Posts: 4
Liked: never
Joined: Oct 24, 2018 2:58 pm
Full Name: Stefan Raudonis
Contact:

Re: GFS for primary backup jobs

Post by raudi »

For me too! Most customers have their backup server in a different fire area and make a replica to other site, they want only the last few backups for DR at a other location but the most backups for faster restores locally. This is currently not possible...
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by Gostev » 2 people like this post

So I wanted to ask everyone here: if it helps us to deliver this functionality sooner, would it be OK to only have GFS available for backup modes with periodic synthetic or active fulls - at least initially? Because then, all functionality comes down to NOT deleting some backup files during the retention policy processing.

This means it won't be available for forever forward incremental backup, like it is available with Backup Copy jobs (which are effectively forever forward incremental). Or for reversed incremental backup. There are significant extra complexities there, since these type of chains do not have "native" fulls - they need to be created. And this functionality is not easily portable to primary jobs.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: GFS for primary backup jobs

Post by mkaec »

My vote is "yes". I will admit that I may be slightly biased, though, as my jobs are configured with periodic active fulls already.

Thinking about this, I think it would be less helpful to implement it for the forever forward incremental or reversed incremental modes. One of the arguments from the crowd that is doing periodic fulls is that they have to implement BCJs to re-create what they already have because it is being thrown away. If you have to re-create anyway when in forever forward and reverse incremental, the advantage seems to go away (except for when ReFS block cloning is being used).
GreenAlpha55
Enthusiast
Posts: 32
Liked: 3 times
Joined: Oct 25, 2018 2:20 pm
Contact:

[MERGED] Backup copy jobs and Fast Clone

Post by GreenAlpha55 »

I'm surprised Veeam does not allow GFS on Backup jobs. Because they don't, it's impossible to adhere to the 3-2-1 rule. Instead, you adhere to the 4-2-1 rule.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Backup copy jobs and Fast Clone

Post by Shestakov »

GreenAlpha55, we will consider it as a request to GFS on backup jobs. Thanks!
hashescanthide
Lurker
Posts: 1
Liked: never
Joined: Nov 05, 2018 7:05 pm
Full Name: HASHES CANTHIDE
Contact:

[MERGED] Feature Request: GFS option in Backup Job Configuration

Post by hashescanthide »

Relatively new to Veeam but have found that it would very nice to have some additional options with the scheduling:

GFS Backup Job type where we can build in GFS options without creating multiple jobs that require extra management and time in looking at success/failure reports.
DGrinev
Veteran
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: Feature Request: GFS option in Backup Job Configuration

Post by DGrinev »

Hi HASHES and welcome to the community!

Thank you for the feature request!
Your request is noted.
derda05
Novice
Posts: 7
Liked: never
Joined: Mar 01, 2013 7:34 am
Contact:

Re: GFS for primary backup jobs

Post by derda05 »

Hi,

I have several customers, who would benefit a lot from that feature.

Thanks,

Dennis
tturnipseed
Lurker
Posts: 2
Liked: 1 time
Joined: Jun 06, 2018 6:39 pm
Full Name: Tate Turnipseed
Contact:

Re: GFS for primary backup jobs

Post by tturnipseed » 1 person likes this post

Gostev wrote: Oct 24, 2018 8:37 pm So I wanted to ask everyone here: if it helps us to deliver this functionality sooner, would it be OK to only have GFS available for backup modes with periodic synthetic or active fulls - at least initially? Because then, all functionality comes down to NOT deleting some backup files during the retention policy processing.

This means it won't be available for forever forward incremental backup, like it is available with Backup Copy jobs (which are effectively forever forward incremental). Or for reversed incremental backup. There are significant extra complexities there, since these type of chains do not have "native" fulls - they need to be created. And this functionality is not easily portable to primary jobs.
For me, the biggest benefit comes from the space savings that ReFS offers when doing synthetic fulls. I think it is perfectly acceptable to only make extended retention rules for primary chains that include periodic fulls.
mma
Service Provider
Posts: 111
Liked: 21 times
Joined: Dec 22, 2011 9:12 am
Full Name: Marcel
Location: Lucerne, Switzerland
Contact:

Re: GFS for primary backup jobs

Post by mma » 1 person likes this post

Gostev wrote: Oct 24, 2018 8:37 pm So I wanted to ask everyone here: if it helps us to deliver this functionality sooner, would it be OK to only have GFS available for backup modes with periodic synthetic or active fulls - at least initially?
Yes Gostev, this would help a lot.

As mentioned before, if you have e "fat" OnSite Backup Server and small OffSite / cloud Repository as "insurance" then you need your GFS OnSite. Right now you need a second OnSite repository and waste a lot of space for redundant data...
Post Reply

Who is online

Users browsing this forum: Amazon [Bot], ante_704, Bing [Bot] and 170 guests