CopyJob and the FULL

[GianlucaCroci]

I've a question regarding the "Immediate Copy" CopyJob. I love this option, but I don't quite understand how it works for FULL.
In the "Target" tab I've set "Weekly backups" on a certain day of the week, which doesn't correspond to the same day as the FULL of the backup to which it refers, and in fact every so often the CopyJob does not do the FULL but continues with the incrementals.

We've a 14 day Retention Policy, and I'd like to avoid creating the FULL (oldest save in the chain) to maintain the 14 day chain. Possible? How?

Many thanks in advance

[PetrM]

Hello,

The backup copy job has its own backup chain structure which is totally independent on the source backup chain. By default, backup copy works in forever incremental mode, therefore you can just set 14 days in the retention policy settings. Also, I'm not sure I fully understood what do you mean when you say that the backup copy job does not create GFS backups sometimes? I'd suggest to refer to this page for more examples of GFS schedule.

Thanks!

[foggy]

Hi Gianluca, GFS mechanism should come into action when the number of restore points in the regular part of the backup chain reaches the number specified in the job settings. Can you confirm this is the case?

[GianlucaCroci]

Thanks all.

I've cases where I've configured the application to do a FULL on the Monday (Restore Point to keep 14, with 2 Weekly Backup), but now I've just one FULL (March 12 2021) and 13 Incrementals.

I don't know if the problem is due to an excess of jobs performed in the day, which cause the FULL window to be lost because the copy starts not on the preset day but a few hours after midnight of the next day.

I see that if I set 14 days of retention, the oldest save is always a generated FULL, because it carries the date of the current day. Do you mean this?

Thanks again

[PetrM]

Hi Gianluca,

If the regular chain has 14 restore points and backup copy still does not create Weekly backups by some reason, then I'm not sure that we could figure it out based on forum posts. Perhaps, it would make sense to let our support team to have a closer a look at your environment.

Thanks!

[foggy]

I've cases where I've configured the application to do a FULL on the Monday (Restore Point to keep 14, with 2 Weekly Backup), but now I've just one FULL (March 12 2021) and 13 Incrementals.

I mean that the job needs to reach its regular retention first (14 restore points, in this case) and then the full backup should reach the day specified in GFS retention settings (Monday). If this has already occurred and the GFS restore point hasn't been created, then it is time to ask our engineers for a closer look at your setup.

[GianlucaCroci]

Before opening a case for support, I'd like to be sure that it doesn't depend on us or on my wrong configuration. That's why I wrote here.

Since every now and then, especially on Sunday after Monday, several CopyJobs queue up because it starts to "merge a full" and the "rest of the world" freezes, so I'd like to investigate first.

As soon as I'm sure it's not up to us, I'll open a case.
I hope before moving on to 11

thanks again everyone

[foggy]

Why does it freeze? Are you limited on the repository slots?

[GianlucaCroci]

I've limitations on both the proxy gateway and the Repository.
On the Gateway server I can have at most 14 concurrent tasks, while on the repository there are at most 12 concurrent tasks.
Too little?

However by freeze I mean that if other CopyJob servers are running, they continue until those servers are finished, but then the other CopyJob or other servers of that CopyJob remain in Pending. All the resources seem to take them for the full merge

[foggy]

Whether it is too little or not depends on the available CPU cores. Each backup chain merge process takes a separate repository task slot, so provided per-VM backup chains are enabled, each VM will require a slot, and all the repository slots, in theory, may be taken by them.

[Taran]

I see a note "I hope before moving on to 11" which indicates that we're talking v10 or earlier Backup copy job behaviour. Info about Backup copy job in version 10.

There were some significant changes to how the backup copy job works in v11, which is why some of the information provided doesn't seem to match with the behaviour seen. It might be easier to upgrade to v11 and understand the new Backup Copy Jobs, instead of trying to understand both the old and new.

Version 10 and earlier Backup Copy Jobs:
There is a forever forward incremental chain of backup files which contains the current data (The number of days/restore points chosen). The merge process that you see which is taking a long time is taking the oldest increment and folding it into the associate full backup, pulling the chain forward by one restore point. The full backup file still contains data from the oldest restore point in the chain, it's just been updated to be one restore point newer. If the repository doesn't have enough CPU cores to handle the number of tasks, this can seem to take a very long time, as the job will wait for resources to become available. (For some storages it may be beneficial to put a gateway server near the repository to provide the necessary processing power.)

The way that GFS works is that the necessary restore point in the forever forward chain will be marked with a "GFS mark". When the full backup reaches this marked restore point, the next run will generate a new full for the forever forward incremental chain, leaving behind the old full backup as the GFS restore point, so you don't see the GFS restore points "appear" as separate files until later than expected.

Version 11 Backup Copy Jobs:
If GFS is not enabled you will see a forever forward chain of backup files which contains the current data (The number of days/restore points chosen). There will be a merge process each run of the job once the desired number of restore points are created in the repository, to keep the chain moving ahead.

If GFS is enabled you will see forward incremental chains of backup files, with the full backups being synthetically created on the day selected for GFS. There will be no merge process, but a weekly process to build a full backup.