Comprehensive data protection for all workloads
Post Reply
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

GFS copy full restore point selection

Post by skrause »

So I have a copy job that is set up as a GFS job that runs daily and keeps 5 weekly archive backups. I have "read entire restore point from source" checked.

All of my source jobs run active fulls every week, and, in the case of one server, that full is what I absolutely have to have archived by the GFS job as this server is where our DBA copies the full backups of our critical SQL servers for the week. That source job runs at 2am with the active full happening Sunday morning (completes by 3am). I have set my backup copy job to archive weekly and set the restore point schedule to look for Sunday 0400. The sync interval for the copy job starts at noon.

Looking at the report for the job (and the size of the VIB files on Sunday), it looks like the job is actually running the full copies on Saturday at noon instead of Sunday at noon which I thought it would be doing with the restore point selection I made.

How do I get a GFS copy job to run its archive full on the day I specify?
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

On Saturday at noon there's no full backup created on source yet, right? Does it create full backup (VBK) on Sunday then?
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

On the source job(s)?

Correct, the fulls for all of the jobs that are source in this particular GFS job run an active full overnight (between 7pm Sat and 4am Sun) to create .VBK.

All of the most recent points on the source jobs as of noon on Saturday are the .vibs from Friday "night".
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

So is VBK created by the the copy job during the next cycle (on Sunday)?
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

No, a VBK was created on Saturday at noon, then there is a VIB created on Sunday at noon which is very large.
Steve Krause
Veeam Certified Architect
rbrambley
Veeam Software
Posts: 481
Liked: 57 times
Joined: Jun 16, 2009 1:23 pm
Full Name: Rich Brambley
Contact:

Re: GFS copy full restore point selection

Post by rbrambley »

I think what you are seeing is the daily .vbk veeam has to create by default. For example, if understand your settings from above, you have a BCJ with 0 dailys and 5 weeklys. Lets say you start this job on a Fri.

Fri - the BCJ witll create a new daily .vbk
Sat - the BCJ will create a temporary .vib, merge with the existing .vbk, and delete the .vib.
Sun - same as SAT and this .vbk becomes your first Weekly

You have 1 .vbk at this point

Mon - same as Sun, but Veeam keeps the 1 Weekly .vbk and starts a new daily .vbk

And so on, until you eventually have 5 weekly .vbks and 1 daily .vbk
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

I am copying the entire from source, there are no transform operations being done. When it reaches the point at which it copies the new full, it just deletes the VIB files per retention.
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

I suggest reviewing the setup with support.
rbrambley
Veeam Software
Posts: 481
Liked: 57 times
Joined: Jun 16, 2009 1:23 pm
Full Name: Rich Brambley
Contact:

Re: GFS copy full restore point selection

Post by rbrambley »

Veeam BCJs don't copy the source backup files as is. BCJs read the data from the source files and then use the temp .vib and merge as I described. It's a common misunderstanding, but BCJs use the same data, but create their own new chain of files.

It gets really confusing and unpredictable when actual job windows overlap the sync schedule, and it's hard to follow what you are seeing exactly in these forums. Your best bet is to open a ticket with Veeam Support so they can see your settings and what's actually happening.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

Steve uses active fulls with backup copy jobs, so merge is out of the picture. Anyway, it's better to troubleshoot this with support, indeed.
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

Ok Thanks.

Since 9.5 came out since I was last in the office on Wednesday morning, I guess I will hold off on sending in a ticket with support until I have that installed and can see if the issue resolves itself.
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

Sounds reasonable. Keep us informed on the results.
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

So the good news: It looks like the copy job has figured out the full restore point selection on its own after the upgrade to 9.5.
The bad news: It ran a full copy on both Saturday and Sunday which scares me a bit for my retention policy if it decides to do the same thing this weekend as I only have a 1 week "buffer" in my retention settings vs policy (I keep 5 weeks, we are only required to keep 4).

Hopefully this was a one-time adjustment. I will watch it Saturday/Sunday and will open a ticket then if the double-full happens again.
Steve Krause
Veeam Certified Architect
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

So it looks like 3/4 of my jobs are now, after the update to 9.5 running an active full on BOTH Saturday and Sunday which has just destroyed my retention policy as well as created a huge cramp on my disk space. In each case, the restore point to copy in the job settings is between 0200-0400 on Sunday. The only job I have that copies only once is set to Saturday 2200 as a GFS point, but it is doing the Full copy on Saturday morning (8 am) so it is still selecting the point to copy in a non-intuitive fashion.

Ticket 01994952
Steve Krause
Veeam Certified Architect
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

So support at this point is saying that it is being caused by a "synthetic operation" happening on Saturdays. When I told them I use Copy Full they told me it was in the source jobs, which are weekly active fulls.

I am in a situation where this is obviously an issue with how Veeam selects when to run with the restore point selection logic for GFS copy jobs. Just getting this up the chain where someone can agree that it is the issue is the problem.

Usually I would roll with it while a fix is worked on but I will be out of the office for the rest of the month (vacation + organization offices closed) starting the middle of next week so my very manual workarounds to make sure I keep my retention while not running out of disk space are not feasible.

I guess I am going to have to create additional copy jobs using a different day to copy the full and run a longer retention time.
Steve Krause
Veeam Certified Architect
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: GFS copy full restore point selection

Post by skrause »

Ok, so there is definitely a bug with restore point selection for GFS that creates double-active fulls to run when you have copy entire source selected.

It looks like, if you have a restore point selection that is on the next day, but before the next syncronization interval, the copy job runs on both days. If you have the restore point set to the same day as your copy interval starts (but after the start of the interval) it pulls it just on that day.

So in my case, my copy interval on one job is set to sync at 8AM. My GFS point selected was originally at 4AM Sunday. When its sync interval on Saturday started at 9am it would run a full instead of an incremental because the GFS point was during its copy interval. Then, when it started the next interval on Sunday at 9am it would run a full because the GFS point was on the same day of the week. I "resolved" the issue by setting my GFS point to be on Sunday at 10AM before I left the office on Friday. When the job ran yesterday it ran an incremental and today it is finishing off its full.

I still find it worrying that when I set a "choose the restore point closest to 2200 on X-day" (default I think) it does not run the Full AFTER that point in time but before.

But at least I have fixed my issue where my retention was being shot to hell because it was running a full two days in a row.
Steve Krause
Veeam Certified Architect
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: GFS copy full restore point selection

Post by foggy »

Thanks, Steve. I've discussed this behavior with QA and they are aware of this issue, it is planned for addressing in the next version. So far please continue using the workaround.
Post Reply

Who is online

Users browsing this forum: ante_704, mibrown9954, mrmccoy007 and 299 guests