GFS for primary backup jobs

Availability for the Always-On Enterprise

GFS for primary backup jobs

Veeam Logoby newfirewallman » Tue Jan 20, 2015 12:11 pm

I love your product it has so much to offer. One annoyance though is the relationship between the backup job vs the backup copy job. I really like the the file retention portion of the backup copy job and how it related to the backup job, but i don't like how the copy job retention forces me to have a minimum of 2 copies. In my scenario i have a lot of wasted space having duplicate backup data, just to accomplish a retention policy (my setup tape plays a part also). I would love to see that have the ability to be zero and only copy data from the primary job when it meets requirements from the retention policy (or add retention policy to primary job with secondary repository location and settings). This would improve the use of Veeam greatly and reduce space. The second feature that would be nice is on the retention policy (GFS) is have those backups not be full but incremental with, transform/reverse depending on the GFS policy to save space as well. Third feature would be the ability under backups/disk and in a job properties have the ability to remove from disk there a specific job vs all jobs. Sometimes there are needs to cleanup and gain space back where i would like to manually remove a job or two especially when a weekly and monthly job fall on the same day or very close. I would love to received some real honest feedback on these request as i feel they would proved outstanding value to an already great product.
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request and Review

Veeam Logoby foggy » Tue Feb 03, 2015 5:06 pm

newfirewallman wrote:but i don't like how the copy job retention forces me to have a minimum of 2 copies. In my scenario i have a lot of wasted space having duplicate backup data, just to accomplish a retention policy (my setup tape plays a part also).

But this is still a backup copy job that is meant to create a copy of the VM backup in some secondary location.

newfirewallman wrote:I would love to see that have the ability to be zero and only copy data from the primary job when it meets requirements from the retention policy (or add retention policy to primary job with secondary repository location and settings).

So basically you're talking about GFS retention for the regular backup jobs here, right?

Anyway, thank you for the feedback, always appreciated!
foggy
Veeam Software
 
Posts: 14337
Liked: 1054 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request and Review

Veeam Logoby Bunce » Wed Feb 04, 2015 2:54 am 1 person likes this post

Proper GFS rotation as implemented in a number of other products was requested for years and avoided, partly due to Veeam's use of file based storage which makes it difficult to implement on the primary copy.

Very disappointing that when it was finally bought in that it forced us to use a second copy. The continued 'we know better than your business - you must keep 3 copies', while valid in some businesses, is simplistic, annoying and some might say arrogant.

Let your customers decide how they wish to implement multiple copies and provide us the flexibility to implement it how we wish. GFS shouldn't be dependant on it.
Bunce
Expert
 
Posts: 254
Liked: 8 times
Joined: Fri Sep 18, 2009 9:56 am
Location: Adelaide, Australia
Full Name: Andrew

Re: Feature Request and Review

Veeam Logoby newfirewallman » Wed Apr 01, 2015 7:22 pm

Yes i 100% agree. I would like GFS retention on the regular backup as an option. Or at a minimum creating the GFS retention from the original to a second repository, but without having to create the data so many times.
In my example i might have a 10TB file server that is backed up in a primary backup and i want to use GFS, now think how much disk space is required or wasted with duplicate data. If i have the original in primary repository at 10TB it will then create a minimum of 2 in my GFS repository 10TB plus 1 incremental, and then when it creates the first weekly or monthly etc. another 10TB. That is a very inefficient use of space.
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

[MERGED] Feature Request

Veeam Logoby SE-1 » Tue Apr 07, 2015 2:36 pm

I have a request for a new feature.

Can the GFS retention scheme can be included in normal backup jobs?

To have GFS retention you now have to work with Backup Copy Jobs.

Backup Copy jobs are not optimal using NAS devices as these devices have SATA drives and some of them on top deduplicated & compressed data.
It seems more logical & faster to transfer a new active full from the primary production storage.

We have a tier 1 storage with 96 SAS disk (48 10K disks & 48 15 disks) & 8 SSD drives, which we backup to a deduplication appliance with 27 SATA disks (50TB) netto.
We achieve very fast backups using multiple VEEAM proxy's in combination with a 10Gbit back bone and we are very happy about it.
Also the syntethic fulls are performing great.

We have configured backup copy jobs to have a retention of 12 months, but this is slower as it reads from the sata disks and this in a sequential manor.
In case a de-duplication appliance is used, it also needs to re-hydrate & uncompress the data before it is send from backup target 1 to backup target 2.

Our production storage achieves much more read performance for an active full backup as a NAS device with SATA drives ever can achieve, which is logical as the tier 1 storage has more & faster disks, and on top SSD drives.

When we check the stats from our production storage & backup targets, we notice that our tier 1 storage is doing nothing during the weekend and our backup targets are working as crazy for the backup copy jobs.
The difference of snapshot sizes for an incremental or full backup on our production virtual machines during the weekend is neglectable.

We see the same for our customers.
More then 80%-90% of our customers have very powerfull TIER1 storage boxes, which have very low performance usage during the weekend.

Also the de-duplication appliances would only store (and replicate) the changed blocks during an active full backup.

Could the GFS retention scheme be build in in normal jobs?

In this way the customer can decide wether to use backup copy jobs to have less impact on the production storage or use the GFS retention in normal jobs.

Thank you
SE-1
Novice
 
Posts: 8
Liked: 1 time
Joined: Tue Apr 07, 2015 1:42 pm
Full Name: Dirk Slechten

Re: Feature Request and Review

Veeam Logoby newfirewallman » Thu Apr 09, 2015 11:13 am

I have been saying the same thing. Call it copy/retention/or normal backup i don't care, but have it integrate better with GFS and another Repository. Save space and IO
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request and Review

Veeam Logoby marco.horstmann » Fri Apr 10, 2015 10:36 pm 1 person likes this post

Hi,

Poul has published an article which could be a solution for you.

http://poulpreben.com/active-full-backu ... -copy-job/

It requests a new full backup from primary backup. Read the blog post maybe it's something you seeking for.

Regards
Marco
Marco Horstmann
System Engineer @ Veeam Software

@marcohorstmann
https://horstmann.in
VMware VCP
NetApp NCIE-SAN for 7-Mode and Clustered Ontap
marco.horstmann
Veeam Software
 
Posts: 265
Liked: 39 times
Joined: Wed Dec 31, 2014 3:05 pm
Location: Hannover, Germany
Full Name: Marco Horstmann

Re: Feature Request and Review

Veeam Logoby newfirewallman » Wed Apr 22, 2015 6:21 pm

That is almost what i want...Almost

Ideally though why even copy the full backup if it isn't needed for the archival GFS retention job. In my case this could be 10TB of data to copy over and then delete. This problem is compounded even more when it would copy over the 10TB from my primary repository, to the secondary used for archival (with dedup) another 10TB, then if it is time to great a GFS job it seems to copy it all again while still keeping the copyjob 10TB. This means i always need to have a large excessive of freespace, plus a lot of "extra" IO that isn't needed.

At least with this script it can save some of the IO between merging the jobs which is nice, but it would be really really nice if it could be handled via the GUI and without the extra write and copy.... would greatly prefer it to just come from the source job/repository when needed.
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

[MERGED] Feature Request - Keep Active fulls as GFS

Veeam Logoby timmi2704 » Mon Jun 01, 2015 2:11 pm

Hi again :D

I see the point in having a backup copy job which maintains the GFS retention policy based on restore points instead of backup files.

But under certain circumstances, I see a possibility of optimizing this process which would be of significant benefit for a job setup like mine.
Each backup job is set to perform active fulls on a weekly base. While trying some different job settings, this is the one which seems to suit our needs the most.
I would really like to have the possibilty to automatically keep some of the weekly active fulls as my "weekly" or "monthly" backups as long as they were successful.
In case of unsuccessful weeklies, the following incrementals might be kept as well so that there is a successful backup of each VM in the job.

Is this far-fetched or would this be of use for anyone else as well? :)

Thanks for having a look at this.
timmi2704
Enthusiast
 
Posts: 78
Liked: 3 times
Joined: Tue Jan 14, 2014 10:41 am
Full Name: Timo

Re: Feature Request and Review

Veeam Logoby foggy » Mon Jun 01, 2015 3:02 pm

Yep, we've already seen similar requests, thanks for the feedback!
foggy
Veeam Software
 
Posts: 14337
Liked: 1054 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request and Review

Veeam Logoby Shestakov » Mon Jun 01, 2015 3:07 pm

Hi Timo!
timmi2704 wrote:In case of unsuccessful weeklies, the following incrementals might be kept as well so that there is a successful backup of each VM in the job.

If there is an unsuccessful active full, there is no need to keep making increments.
Thanks!
Shestakov
Veeam Software
 
Posts: 4529
Liked: 362 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Feature Request and Review

Veeam Logoby timmi2704 » Tue Jun 02, 2015 9:19 am

Thanks, foggy, for merging my request :D

Shestakov wrote:If there is an unsuccessful active full, there is no need to keep making increments.

Hi Nikita.
I was talking about some specific VMs which failed in the active full and its retires but were successful in the next incremental backup. In this case, the weekly full and the incremental backup would be needed in order to have a successfull "full backup" for all VMs. Correct me if I'm wrong :)
timmi2704
Enthusiast
 
Posts: 78
Liked: 3 times
Joined: Tue Jan 14, 2014 10:41 am
Full Name: Timo

Re: Feature Request and Review

Veeam Logoby foggy » Tue Jun 02, 2015 3:06 pm

You're correct, Timo.
foggy
Veeam Software
 
Posts: 14337
Liked: 1054 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] [Feature Request] GFS in simple Backup Job

Veeam Logoby VladV » Sat Aug 22, 2015 12:06 pm

I think I've seen that this was a request some time ago but I am having difficulties in finding it.

Anyway, it would be nice to have the GFS functionality from Backup Copy in a simple Backup Job.

Using server 2012 R2 Deduplication we don't have the need for 2 onsite storage targets: one for fast recoveries and one for long term storage. We can combine them into one and schedule dedup only for older files. This way we manage to have a 14,5TB RAID10 (16 disk) storage hold 37TB of data and still have the latest month "undeduped". We currently have 90 restore points per each job, one per day, and one active full per week (classic backup). If we had GFS in the simple Backup Job, we could have spared more space by eliminating incremental restore points (which have a low dedup ratio - being incremental :)).

If you guys also believe this could be useful in other scenarios, please consider including it in a future release.

Thanks,
Vlad
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Feature Request and Review

Veeam Logoby Shestakov » Sat Aug 22, 2015 8:16 pm

Hello Vlad,
The best practice is to have at least 2 copies of backups, one of them onsite made by backup job and another offsite or on the tapes made by backup copy or backup-to-tape job. There is a GFS for backup copy job and we are adding GFS option for backup to tape jobs in v9.
Since basic backup job is not considered as a job making historical backups, GFS is not going to be an option for them.

Do you make copies of your backups?
Thanks!
Shestakov
Veeam Software
 
Posts: 4529
Liked: 362 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: rkovhaev and 45 guests