Backup Copy job appears to not respect GFS setting

[Wayneywoo]

Hi everyone, hope we're all having good days!

I have a backup job:
Retention 7 days
Daily (Hourly 7am - 7pm)
GFS - Weekly (5 weeks)
GFS - Monthly (5 months)

This is happening as expected, I see RP's going back to the start of the year (4th Jan) for past few months, then some weeklies and then dailies.....all seems to be correct

I have a Backup copy job (Named Monthly Copies) which has the reg entry, BackupCopyMirrorAll set to copy all points from start of the original job with a view to keeping just the monthly points. This seems to start as expected, it goes back to the start of the year and copies *all* points up to the latest point but then, for some reason, retention removes most of the previous points and doesn't appear to respect the GFS - "keep monthly points for........"

To illustrate this, prior to 27th April, mid-day ish, there were around 40 RPs in the backup copy job going back to 4th Jan this year (same the the backup job), this being when the original backup job was configured and then following this, we had a monthly for Feb, March and April with some weeklies and dailies copied from the original job. You would expect all points necessary to keep the chain intact would be copied (seemed so) and when retention is ran on the original backup job, the copy job does what?

Then, mid-day on the 27th, around 37RPs were removed, in the job log in the VBR client, there is around 40 lines of "Removing ...... per retention policy". Only RPs from the 27th April are kept (4th Jan up to 27th were deleted) and so the GFS setting in the backup copy job for "keep monthlies for .... months" doesn't appear to have been respected. No monthly RPs exist in the copy job.

Backup Copy has Retention policy of 2 RPs with GFS set, Keep Full Backups for archival purposes, 1 weekly, 12mothly

Just seems like the Backup Job appears to be working as expected but for some reason the copy job copies all RPs you would think are necessary (when ran) and then at some point, something triggers retention on the copy job and removes everything you think you should be keeping due to the GFS config?

Anyone able to help me understand what is going on and why?

Many thanks!

[david.domask]

Hi Wayneywoo,

> BackupCopyMirrorAll

I think this registry value is causing the confusion -- Backup Copies, even with the registry value, are not a simple 1:1 file copy of the source backups to the target repository. Backup Copy is a "backup of a backup", so data is read from inside the original restore points and a new, independent backup copy chain is created on the target repository.

By default, Backup Copies process:

- Most recent restore point on initial run
- (for immediate mode) All new subsequent restore points as they appear

What BackupCopyMirrorAll is change that "most recent restore point" condition to be "all restore points", so each point will be processed by the backup copy job. However, the Backup Copy backups still follow the retention set in the Backup Copy Job. My guess is that your retention settings in the backup copy were set lower than the chain length, and the end result was after the job ran, retention saw it needed to act. Remember, it's just the backup data that is copied over, not the source job retention settings.

Long-term what is your goal? You want to have a 1:1 copy of the source (original) backup on another repository?

I would recommend the following:

1. Moving forward, create a Backup Copy job when you create a primary backup that you need secondary copies of. Configure the backup copy retention (including GFS) as desired, and the Backup Copy will stay in step with the source backup -- same data, just different backup jobs and backup chains.
2. For existing source jobs that need a backup copy, consider doing a Copy Backup to get a copy of the existing backups, and create a backup copy to start the Backup Copy chain, configure retention as desired

(Hint: If relevant, as of Veeam Backup and Replication 12.3, you can create a backup copy from a backup copy, so if you need further copies or want to use an existing backup copy as a source, you can)

[Wayneywoo]

Hi David,

Thanks for replying and the detailed post!

What the goal is, a source job started a while ago to one NAS, a copy of this sent to another NAS with retention on the copy set to keep just monthlies.

I found the reg key because a copy job doesn't go back to the start of the whole original backup chain but rather to the last full necessary for the chain to start and then create copies of RP's from that point. That's my understanding, the reg key was needed in order to get the copy to go back to the very start?

The original backup job keeps dailies, weeklies, monthlies as configured and then the backup copy to keep just monthly copies but ideally, from the start of the year.

The backup copy job had its own retention settings set and reg key existing before being run for the first time and this was left so, as I have done already, when creating a new backup copy job and adding the job name to the reg key and running the job, the backup copy does what I expect, goes back to the very start, 4th Jan.

Ultimately, we have a job already existing that has its first full (Monthly) 4th Jan with monthlies, weeklies and dailies from this. I want to copy this data to another repository and keep just monthlies from the very beginning and not just the latest.

Maybe I run the copy job to get everything copied from the start with the reg key set as necessary and then, create a new copy job and map it to the chain already existing on the second repository?

NAS 1
Chain starts 4th Jan with dailies, weeklies and monthlies as set

NAS 2
To contain a copy going back to 4th Jan keeping only monthlies

Thanks for the help!

[david.domask]

Hi Wayne,

You're very welcome for the answers so far.

> I found the reg key because a copy job doesn't go back to the start of the whole original backup chain but rather to the last full necessary for the chain to start and then create copies of RP's from that point. That's my understanding, the reg key was needed in order to get the copy to go back to the very start?

Correct, however, it's important to keep in mind that Backup Copies produce their own backup chain with the source backup as a source. So the Backup Copy will have its own retention, GFS, etc and _the backup copy retention settings_ will be applied to the backup copy chain -- it does not import the source chain retention. Effectively, the registry value allows the Backup Copy to process all existing restore points, but the resulting backup copy backups will follow the Backup Copy retention settings.

>I want to copy this data to another repository and keep just monthlies from the very beginning and not just the latest.

How many monthlies right now? if it's feasible, I would use Export Backup on these to make a copy on the remote site, and let the current Backup Copy do its job and create its own GFS archives as per normal operation. That way you have auto-expiring copies of the monthlies on the secondary site, and the Backup Copy job will produce archival points as per normal operation as the source job creates new points.

So basically copy the existing monthlies as a one-time thing to have the copies, and let a Backup Copy job run as normal creating a backup copy from the source job.

[Wayneywoo]

Hi David, thanks for the quick reply!

>> I found the reg key because a copy job doesn't go back to the start of the whole original backup chain but rather to the last full necessary for the chain to start and then create copies of RP's from that point. That's my understanding, the reg key was needed in order to get the copy to go back to the very start?

>Correct, however, it's important to keep in mind that Backup Copies produce their own backup chain with the source backup as a source. So the Backup Copy will have its own retention, GFS, etc and _the backup copy retention settings_ will be applied to the backup copy chain -- it does not import the source chain retention. Effectively, the registry value allows the Backup Copy to process all existing restore points, but the resulting backup copy backups will follow the Backup Copy retention settings.

Yes, this was what I understood and yet, with the retention settings set in the backup copy job, it doesn't appear to follow what is set and just copies every point from the source backup and then, at some point, a whole bunch of RP's are deleted.

>>I want to copy this data to another repository and keep just monthlies from the very beginning and not just the latest.

>How many monthlies right now? if it's feasible, I would use Export Backup on these to make a copy on the remote site, and let the current Backup Copy do its job and create its own GFS archives as per normal operation. That way you have auto-expiring copies of the monthlies on the secondary site, and the Backup Copy job will produce archival points as per normal operation as the source job creates new points.

>So basically copy the existing monthlies as a one-time thing to have the copies, and let a Backup Copy job run as normal creating a backup copy from the source job.

I wanted to use the backup copy functionality with the reg key set in order to take advantage of Block Cloning on ReFS. If I export and create a backup copy job, the exports won't take advantage of block cloning and so the required storage will increase.

It's strange that the retention settings in the copy job don't get followed. It I delete all the RP's and start the job again, I get a mirror of all points from the original job and then something triggers the retention and deletes a whole bunch of points in the copy chain....

[david.domask]

> Yes, this was what I understood and yet, with the retention settings set in the backup copy job, it doesn't appear to follow what is set and just copies every point from the source backup and then, at some point, a whole bunch of RP's are deleted.

That part seems unexpected, I would advise open a Support case and let Veeam Support review the logs and see why retention applied. I'm guessing it's about the simple retention settings being too low, but Support will be able to tell from the debug logs.

[Wayneywoo]

Thanks for your help David, I've submitted a case 07686381

Will update when possible