Clarification how Backup Copy GFS retries work in v11a after failures

[kantona]

Hello all,

I hope someone can help me clarify a difference in GFS processing between v11a and the old v10.

In short, my setup is like:
1). Daily Backup jobs with 31 restore points (RPs) retention. Not using GFS in the Backup job currently.
2). Daily Backup Copy jobs to copy all new RPs, with 7 RPs short term retention. GFS retention set as 12 Monthly and 7 Yearly.

In v10 as most of us would know, the above means in the Backup Copy I would have a 7 RPs chain (VBK + 6 VBIs). As the chain moves forward it would create a synthetic full for each month. If it fails to create it on first try, it continues to try on next executions of the Backup Copy job until it manages to create the Monthly VBK. During this period you can have more than 7 RPs in the chain but that gets sorted out once all processing is completed.

After moving to v11a, I am afraid some monthly backups will be lost (not created due to circumstances). As we know the GFS processing changed in v11a and it creates the monthly full on the start of the month, irrelevant of the short term retention chain. The guide also says that if it doesn't create the monthly it will do it on the next execution. So far so good.

But what I am seeing is that if it doesn't create the monthly VBK after 2-3 tries, it gives up. The Backup Copy job continues to add the daily incremental VBIs and does not try to create the monthly VBK any more.

Not only that you end up with long chain in the Backup Copy job (I can live with that short term), but you are not sure if the monthly backup will be created and kept according to retention. For example for one copy job where there were different issues during the monthly VBK processing, currently I have a chain with a full from 27-Apr and incrementals since then. No monthly for start of May, nor for June.
So even if the monthly for July get created correctly, after the processing is completed and the short term chain is deleted, I will end up without monthlies for May and June?? That would be not acceptable from business perspective.

I hope I explained it correctly. I would appreciate some advice if this is normal. As the moment I don't know if I can attribute it to a bug, or I did something wrong. FYI, the problem happened only on one copy job out of three. The other two managed to correctly create the monthly for June so I did not observe this issue.

Thanks in advance.

[HannesK]

Hello,
and welcome to the forums.

But what I am seeing is that if it doesn't create the monthly VBK after 2-3 tries, it gives up. The Backup Copy job continues to add the daily incremental VBIs and does not try to create the monthly VBK any more.

That sounds wrong, because the user guide says

If for some reason the GFS synthetic full was not created in the scheduled day, Veeam Backup & Replication will create the synthetic GFS full after the next run of the backup copy job.

Do you have a chance to upgrade to V12 and see if the problem persists? If yes, Veeam support should check the behavior (you can also check with V11, but if it's a bug, then it will only be fixed in V12)

Best regards,
Hannes

[kantona]

Hello Hannes,

Thanks for the reply. Well actually I can't upgrade to V12 right now yet. But I continue to be worried what will happen at start of July and what will happen to the current chain.

I also opened a case with Veeam about this, case number 06122333, but basically they just confirmed what I could see (VBK not being created because of no free space in the repo at that moment), and when asked directly what will happen to the INC chain they said I will lose it:

In our current situation, once the next Monthly GFS points are created and followed by 6 points, the previous chain (.vbk 27/04 and its increments) will be removed by retention. The points from .vbk 27/04 don't have GFS flags, so they are considered as regular points that are not protected from the removal. You can check the flags that are assigned for the points in the backup properties.

If you wish to preserve the points from May and the beginning of June, the better approach would be to copy the backup chains starting from .vbk 27/04 up to 1/06 to another location. By doing so, you will have these chains available for importing into Veeam in the event of a restore being required. Once you don't need these points anymore, you will remove them manually.

I asked directly whether there is hope that after GFS process creates the 01-Jul monthly backup and as short term retention processing continues (7 RPs short term retention) is there hope that this processing will convert into 01-May and 01-Jun monthly RPs (as it was doing in V10, and because these RPs don't exist right now). But the reply from Veeam Support was a NO. Assuming they understood me correctly. Because at the end of the day, I still have the 12 monthly GFS policy on the Backup Copy job. So any short term retention chain processing should leave monthly RPs too, no?

Thanks for any further clarification you can make.

[kantona]

OK, I got some important piece of information from Veeam Support.

The creation of the Monthly points is retried during the first week of the Month, according to the paramater selected for the GFS Monthly circle : ''Use weekly full backup from the following week of the month : First''. If the job fails to create the Monthly point within the first week (first 7 days) of the month, it will cease further attempts to create it.

In my case I still can't use fast cloning so Exchange backups synthetic full can take 3-4 days. Basically two failed attempts will lead it over the first week cut-off date. And then it doesn't try again.

I still don't know if there is any hope the "short retention" processing to save the day, like explained in the previous post.

[HannesK]

Hello,
the behavior from V10 will not come back in V11 / V12 / vNext, that's correct.

As there is not enough disk space, I'm not sure, what the software should do instead of failing.

I just looked at the case number and with that size you seem to have, I can only recommend to switch to XFS / REFS as soon as you can. V12 (as soon as you can upgrade) allows a migration / move that adds block cloning space savings right after the migration.

Best regards,
Hannes

[kantona]

Hello,
I am already working on deploying few Ubuntu VMs that will have VMDKs formatted as XFS for fast clone support.
If I understood correctly, if I also upgrade to V12 I will be able to move backup chains to the new fast-clone enabled repos and it will automatically use fast cloning without any additional steps being necessary (like active full, or backups defragmentation, etc).
Because in the link you provided, and in the related Moving Backups https://helpcenter.veeam.com/docs/backu ... ml?ver=120 part of the user guide, I didn't see a reference to fast cloning anywhere.
Thanks.

[HannesK]

Hello,
your understanding is correct.

Best regards,
Hannes
PS: physical machines are recommended as repository for security and scalability reasons.