GFS copy full restore point selection

[skrause]

So I have a copy job that is set up as a GFS job that runs daily and keeps 5 weekly archive backups. I have "read entire restore point from source" checked.

All of my source jobs run active fulls every week, and, in the case of one server, that full is what I absolutely have to have archived by the GFS job as this server is where our DBA copies the full backups of our critical SQL servers for the week. That source job runs at 2am with the active full happening Sunday morning (completes by 3am). I have set my backup copy job to archive weekly and set the restore point schedule to look for Sunday 0400. The sync interval for the copy job starts at noon.

Looking at the report for the job (and the size of the VIB files on Sunday), it looks like the job is actually running the full copies on Saturday at noon instead of Sunday at noon which I thought it would be doing with the restore point selection I made.

How do I get a GFS copy job to run its archive full on the day I specify?

[foggy]

On Saturday at noon there's no full backup created on source yet, right? Does it create full backup (VBK) on Sunday then?

[skrause]

On the source job(s)?

Correct, the fulls for all of the jobs that are source in this particular GFS job run an active full overnight (between 7pm Sat and 4am Sun) to create .VBK.

All of the most recent points on the source jobs as of noon on Saturday are the .vibs from Friday "night".

[foggy]

So is VBK created by the the copy job during the next cycle (on Sunday)?

[skrause]

No, a VBK was created on Saturday at noon, then there is a VIB created on Sunday at noon which is very large.

[rbrambley]

I think what you are seeing is the daily .vbk veeam has to create by default. For example, if understand your settings from above, you have a BCJ with 0 dailys and 5 weeklys. Lets say you start this job on a Fri.

Fri - the BCJ witll create a new daily .vbk
Sat - the BCJ will create a temporary .vib, merge with the existing .vbk, and delete the .vib.
Sun - same as SAT and this .vbk becomes your first Weekly

You have 1 .vbk at this point

Mon - same as Sun, but Veeam keeps the 1 Weekly .vbk and starts a new daily .vbk

And so on, until you eventually have 5 weekly .vbks and 1 daily .vbk

[skrause]

I am copying the entire from source, there are no transform operations being done. When it reaches the point at which it copies the new full, it just deletes the VIB files per retention.

[foggy]

I suggest reviewing the setup with support.

[rbrambley]

Veeam BCJs don't copy the source backup files as is. BCJs read the data from the source files and then use the temp .vib and merge as I described. It's a common misunderstanding, but BCJs use the same data, but create their own new chain of files.

It gets really confusing and unpredictable when actual job windows overlap the sync schedule, and it's hard to follow what you are seeing exactly in these forums. Your best bet is to open a ticket with Veeam Support so they can see your settings and what's actually happening.

[foggy]

Steve uses active fulls with backup copy jobs, so merge is out of the picture. Anyway, it's better to troubleshoot this with support, indeed.

[skrause]

Ok Thanks.

Since 9.5 came out since I was last in the office on Wednesday morning, I guess I will hold off on sending in a ticket with support until I have that installed and can see if the issue resolves itself.

[foggy]

Sounds reasonable. Keep us informed on the results.

[skrause]

So the good news: It looks like the copy job has figured out the full restore point selection on its own after the upgrade to 9.5.
The bad news: It ran a full copy on both Saturday and Sunday which scares me a bit for my retention policy if it decides to do the same thing this weekend as I only have a 1 week "buffer" in my retention settings vs policy (I keep 5 weeks, we are only required to keep 4).

Hopefully this was a one-time adjustment. I will watch it Saturday/Sunday and will open a ticket then if the double-full happens again.

[skrause]

So it looks like 3/4 of my jobs are now, after the update to 9.5 running an active full on BOTH Saturday and Sunday which has just destroyed my retention policy as well as created a huge cramp on my disk space. In each case, the restore point to copy in the job settings is between 0200-0400 on Sunday. The only job I have that copies only once is set to Saturday 2200 as a GFS point, but it is doing the Full copy on Saturday morning (8 am) so it is still selecting the point to copy in a non-intuitive fashion.

Ticket 01994952

[skrause]

So support at this point is saying that it is being caused by a "synthetic operation" happening on Saturdays. When I told them I use Copy Full they told me it was in the source jobs, which are weekly active fulls.

I am in a situation where this is obviously an issue with how Veeam selects when to run with the restore point selection logic for GFS copy jobs. Just getting this up the chain where someone can agree that it is the issue is the problem.

Usually I would roll with it while a fix is worked on but I will be out of the office for the rest of the month (vacation + organization offices closed) starting the middle of next week so my very manual workarounds to make sure I keep my retention while not running out of disk space are not feasible.

I guess I am going to have to create additional copy jobs using a different day to copy the full and run a longer retention time.

[skrause]

Ok, so there is definitely a bug with restore point selection for GFS that creates double-active fulls to run when you have copy entire source selected.

It looks like, if you have a restore point selection that is on the next day, but before the next syncronization interval, the copy job runs on both days. If you have the restore point set to the same day as your copy interval starts (but after the start of the interval) it pulls it just on that day.

So in my case, my copy interval on one job is set to sync at 8AM. My GFS point selected was originally at 4AM Sunday. When its sync interval on Saturday started at 9am it would run a full instead of an incremental because the GFS point was during its copy interval. Then, when it started the next interval on Sunday at 9am it would run a full because the GFS point was on the same day of the week. I "resolved" the issue by setting my GFS point to be on Sunday at 10AM before I left the office on Friday. When the job ran yesterday it ran an incremental and today it is finishing off its full.

I still find it worrying that when I set a "choose the restore point closest to 2200 on X-day" (default I think) it does not run the Full AFTER that point in time but before.

But at least I have fixed my issue where my retention was being shot to hell because it was running a full two days in a row.

[foggy]

Thanks, Steve. I've discussed this behavior with QA and they are aware of this issue, it is planned for addressing in the next version. So far please continue using the workaround.