Backup COPY job scheduling changes in V11

[AndraZ]

Hi all. We have some V11 troubles (behavior we can't explain) and support is strugling. So I decided to ask here for ideas.
Case ID: 04861124

After upgrading to V11 we ran into problems with scheduling backup Copy jobs. We have daily Backup Jobs with different settings, that are input to our Backup Copy Jobs. Backup Copy jobs are typically set as forever incremental, that should run every 30 days and should keep 12 backups. This translates to a chain with one full backup and 11 monthly incrementals, with a process of merging last incremental on every job run once a month. We also have GFS enabled for 1 yearly backup, that should create separate independent full backup on January. This all worked fine in last 2 major versions. But right after upgrade when I enabled back those jobs I noticed that they went into execution even though less than 30 days have passed since last run. I dismissed this as a one time thing after update. But now we see that on every Veeam server restart (for windows updates, regular restarts or after some error) it seems that this jobs are starting and doing incrementals, no matter how may days passed from last run. So we now have around 5 extra incremental files on such copy jobs that should not be there. Job logs and additional incremental file dates correspond with server restarts.
One more interesting thing is that even though incremental file count is now more than 12, jobs don't merge old files to full.

These are settings for one example of such Copy Job:
It has Periodic copy (pruning) of a Backup Job, with Copy every 30 days at 4:00.
It has retention policy of 12 restore points
It has GFS for 2 yearly backups on January.
This is file list of the chain:
https://drive.google.com/file/d/16douIC ... 6yXqq/view
Upgrade to V11 was on 17th of May. Up until then chain moved monthly and kept 12 points. From that point on the chain is all over the place and is not merging anymore.

I got this as first response of support, which is a secondary issue, not so important:

In version 11 the long-term GFS retention for backup copy jobs has changed. If a GFS archive is configured for the backup copy job, it will switch from Forever Forward Incremental mode, where the job did merges to keep retention, to Forward Incremental, where it will wait to have enough restore points to meet retention from the most recent full backup (the yearly backup in your case) until it removes restore points behind that full backup.

I kinda get what this means. We used yearly GFS on those 12 month backup copy jobs to move it to another location as independent archive. Veeam made this full for the last January at the point when the oldest file (full) from the 12 month chain passed this January. So we have them for January 2020, but for 2021 they will be created when chain leaves January. This was pre V11 behavior and it seems it can't be replicated in V11.
But let's leave this for later and return to the main issue, scheduling of Backup Copy jobs.

With support I then did this:
- removed last few incremental files to clean up this chain
- rescaned the repository (https://helpcenter.veeam.com/docs/backu ... ml?ver=110)
- removed (forget) the missing files from the restore job window (https://helpcenter.veeam.com/docs/backu ... ml?ver=110)
- deleted original copy job
- recreated copy job with same settings, except for GFS (we turned it off in one job and ON on the other to see if it makes any difference), and with a map to the chain (https://helpcenter.veeam.com/docs/backu ... ml?ver=110)
After enabling the new job new incremental was created, which is OK, because we cleaned up chain enough.
But then we updated windows (monthly rollup) and restarted the server twice, and it was again doing new increments after restart, ignoring 30 days completely. So nothing changed yet.

Any idea, what is going on?

BR, Andraž

[PetrM]

Hi Andraz,

The periodic backup copy identifies recent restore points that must be processed according to the following rule:

Time of restore point creation >= current time – backup copy interval

You may refer to this page on our help center for more examples.

If backup copy does not respect the rule above, then it looks like a technical issue which must be analyzed based on debug logs. You may also escalate the support case if you believe it's not on a right track.

Thanks!

[dome73]

Hi Andraz,
same backup copy policy (every 30 days with 12 rp) and same problem as your after upgrading to v11.
Did you find any work-around to avoid a new backup copy after every reboot?

Thanks

[PetrM]

Hi Domenico,

Currently, there is no fix for this issue but our development and QA teams work on it. Although, I cannot comment on ETA so far. Please ask our engineers to check whether your problem matches the one described above. If it's the case, support can provide you with a workaround.

Thanks!

[JeroenL]

We see something simular after upgrading to V11 P20210525.
We use Catalyst Replication tasks for backup copy of an entire repository.
With V10 every new restore point from a normal backup triggered the Catalyst replication to start, but now I have to manually start the "sync now" for a replication to occur. New files are detected this way.

I already tried to recreate the Catalyst Copy Jobs and tried to alter the schedule from "Any time" to scheduled with everything selected for replication, but this doesn't help. Monday I will create a support case, but this weekend I have to click "Sync now" every few hours to make sure data is copied to the offsite location.

[PetrM]

Hi Jeroen,

Please don't forget to provide us with a support case ID so that we can keep an eye on it.

Thanks!

[JeroenL]

Hello PetrM,

After I contacted support a hotfix was provided that solved our issues.