Suggestion for a different way of handling backups/versions

[werten]

Dear Veeam Support,

I would like to make a suggestion for Veeam Backup & Replication:

In the company I work for, we use Veeam for regular backup of all of our virtual machines. Given our specific backup requirements, we have multiple backup jobs for some these VMs (daily, monthly, yearly, etc.). These are sometimes overlapping/complementary and maintaining an overview (and not needlessly wasting space on the backup location) is rather cumbersome...

At home, I'm using CrashPlan (from code42) for my personal (PC-based) backup; I know, I know, this is something completely different, but they do have a way to setup backups that is very easy: in every job you set up, there is an option to specify how often new backups should run and how many restore points you want to keep for different time intervals; all of this in a single window, which gives you a great overview.

The way you can do this is also very intuitive: you can set to keep daily backups for the last week, weekly backups for the last months, monthly backups for the last years, etc. etc. (freely tunable, of course, but this is how I have set things up).

All of this is done via quickly adjustable sliders, and the backup data is actually reorganized when one changes something or when changes need to be made based on the retention rules that one has set!

Please have a look at this software (the settings discussed can be found under "settings >> backup >> frequency and versions >> configure"), just to get an idea about the interface and logic that is applied.

I would hope that you really consider adopting something like this for Veeam B&R as well: I believe it would help many users to more quickly, more intuitively, and more efficiently setup backups for their systems.

Many thanks!

[Shestakov]

Hello Paul,
There is such an option in VBR, called GFS retention. By design it works with backup copy job. Have you got a chance to work with it?
By the way, we have Veeam Endpoint Free for physical servers and desktop protection.
Thanks!

[werten]

Hi Shestakov,

GFS retention looks interesting and may fit the bill, but I can already tell from quickly looking at the descriptions that it is much more complex to set up than what I described. It would be nice to have a simple interface and for this to work on a single backup location and within the same job settings

Cheers...

[Shestakov]

CrashPlan may look simplier UI because of smaller number of options or just because you`ve got accustomed to it.
In any case we will take into account your suggestion and review their UI.
Thanks!

[veremin]

GFS retention looks interesting and may fit the bill, but I can already tell from quickly looking at the descriptions that it is much more complex to set up than what I described.

I don't think a backup copy job with GFS retention would be more complex than your current approach of orchestrating multiple jobs with different retention settings.

The options might look a bit complicated at first, but all you need to do is to select source VMs, specify how often restore points should be created for them (copy interval) and set number of GFS restore points you want to preserve. That's it.

Thanks.

[werten]

Just noticed that the maximum interval for backup copy jobs is 100 days... Not enough for yearly jobs... Is there any way around this?

What I think would also suit my purposes is to have a "full backup only" job option in the Advanced Settings (in addition to the current Reverse Incremental and Incremental options that can be selected in Backup Mode); I tried incremental with Active Full Backup selected in addition, but this still depends on the availability of previous full backups in the chain and also doesn't allow me to manually run just the full backup if I choose to do so "in between" regular backups. A "full backup only" option would overcome that limitation and to my opinion would make sense for a lot of folks...

Cheers...

[werten]

OK, just saw that I can do the Active Full in between by pressing that button in the Job Control section... My bad...

[veremin]

Not enough for yearly jobs... Is there any way around this?

Just to clarify - are you willing to store 365 daily increments by backup copy job? Thanks.

[werten]

Yes, the idea would be to keep yearly backups of a few servers, for a maximum of 5 years.

In addition to that (and in separate jobs), we have regular daily and monthly backups for a more limited time...

Thanks for any suggestions on the yearlys!

[werten]

Sorry, I read your question incorrectly:

No, the idea is to keep full backups from let's say December 2012, December 2013, ... , December 2016 and so on, deleting the oldest when a new one is added at the end of the year.

In addition to that (and in separate jobs), we have like 12 monthlys and 14 dailys (mostly via reverse incremental jobs) to take care of the more current things.

Cheers

[veremin]

So, in this case why not to create a backup copy job and set it in the following manner:

- Copy every: 1 day
- Restore points to keep: 14 (daily)
- Keep the following restore points for archival purposes: Enabled
- Weekly: 0
- Monthly: 12
- Yearly: 5

These settings should give you exactly what you're looking for.

Thanks.

[werten]

OK, great; thanks for the quick reply! I'll certainly look into these options, which I wasn't aware of...

Some quick questions beforehand:

1. I still need a backup job to do the dailys, right?
2. Since I have these set up as Reverse Incremental Backups (saving me HDD space), and since I believe I actually have 30 dailys, I guess I could use the scheme you suggested but just change the copy interval to 30 days, right? (saving some bandwith )
3. Does it matter that the 30 days won't exactly be 1 month in terms of the archival retention settings? Would you guys consider adding weeks, months, or even years as option for the copy interval settings?

OK, I'll have a look now... Thx.

[foggy]

1. Yes, the regular backup job should be selected as a source for the backup copy job.
2. Yes,you can do that if you do not need daily backups offsite.
3. Monthly restore point will still be offloaded on reaching the day specified in the GFS settings (during the next job cycle after that, actually).

[werten]

Thanks again for the comments!

OK, I had quick look at this and it does look very interesting; in fact, particularly the scheduling interface where you can set these retention policies is something that I initially had in mind and talked about when starting the topic

Have you guys ever considered adding such a page for setting up the backup jobs themselves? It would be all I need, I guess, because the copy job that I'm looking at now (after your suggestions) would just be from one NAS to the next.

One question also to that then: if I were to rely on the Reverse Incremental jobs as sole source for the Copy job, would that be "risky" in the sense that these reverse incrementals all are "synthetic" in a manor of speaking? I mean as in contrast to full backups... Thx.

[foggy]

If you're using periodic health checks and SureBackup, then you're covered.

[werten]

Health checks yes, SB no.

I figure I could also add the "perform periodic full backups" to the source reverse incremental job to decrease the chances of errors getting into the backups...

[foggy]

Yes, you can consider this as well.

[werten]

OK, I'll try to set something up to test a bit. Thx so far on this!

But what do you guys think about my suggestion to add the "advanced backup retention schedules", so to say to, as they exist in the backup copy job directly into the backup job options? Wouldn't that be a much more intuitive way of handling the entire backup retention policy thing, similar to what I suggested from Crashplan? I could imagine that the various locations where one now can add (different!) schedules (multiple locations in backup and copy jobs) can cause a lot of confusion here and there, and may make it difficult for some users to quickly see/predict what will happend when and where...

Thanks for considering

[foggy]

We've had similar requests, but not too many, to be honest.

[clement.cromology]

I tend to agree with the topic. GFS handled by backup copy job is way too awkward when dealing with on-site GFS retention.

Basicaly, you have two jobs when only one should be necessary.

For my needs, for a given perimeter of VMs, I wanted to keep 25 daily backups and 52 weekly backups. In my idea, it should work like that :
- a single job with parameters for 25 daily backups and 52 weekly backups, backups from monday through saturday, incremental forward, active full backup saturday
- daily backup runs its course and a mecanism makes sure that I can go back 25 daily versions
- weekly backups are full backups and marked as "protected" until 52 versions are stored
- everything in a single repository

The way it worked when I tried to use backup copy based GFS :
- a single job with parameters for 25 daily backups, monday through saturday, active full backup saturday
- a backup copy job with a 52 weekly backups retention, running on sunday

First, I had to create a second share because it seems that a backup copy job can't be directed to the same repository as the source backup, and for 25 days I had both the saturday full and sunday full copy stored resulting in an unwanted 2x data footprint. Plus it added unnecessary stress to my backup appliance all sunday reading and writing from one share to another on the same disks.

GFS backup copy makes sense when your GFS retention is off-site and daily backups on site. When it's all on site, it just doesn't in my point of view.

[clement.cromology]

Forgot something : how I dealt with it.

I ended up creating a "daily" job with 25 retention points from monday to friday with active full on friday and a "weekly" backup job every saturday, forced active full with 52 retention points.

Still have two time as much data as I would need since I perform an active full both on friday and saturday but it's still less stress to the storage appliance than using backup copy to deal with GFS retention.

[werten]

@clement.cromology, also check out this thread: veeam-backup-replication-f2/feature-req ... 43-15.html

[readie]

So, in this case why not to create a backup copy job and set it in the following manner:

- Copy every: 1 day
- Restore points to keep: 14 (daily)
- Keep the following restore points for archival purposes: Enabled
- Weekly: 0
- Monthly: 12
- Yearly: 5
.

We tried this, and found that it used large amount of backup space, as each Monthly is a FULL, so you end up with 12 full monthly backups. We moved it to a new BackupCopy job, performed monthly, but incremental (or reverse Incremental I think) which then uses much less space.
So we actually have three jobs - daily keep 7, fortnightly keep 4, quarterly keep 3, and an annual to USB HD for archive.

[werten]

@readie, couldn't agree more! Too many jobs and wasted space for what should be a simple task/policy...

[Shestakov]

You can set up a simple retention and run it once a month, so you will have a chain of full and incremental files instead of fulls only.
However during a month VMs have lots of changes and increments will be relatively big.
Thanks!