Discussions related to using object storage as a backup target.
Post Reply
mcz
Veeam Legend
Posts: 835
Liked: 172 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Location: Rheintal, Austria
Contact:

BCJ / offload destroyed by cancelling BCJ?

Post by mcz »

Hello,

since quite a while I'm having issues and finding bugs in combination with offload/object storage and last week, I've "found" another issue, which is maybe due to the new BCJ copy mode. My support engineer (T2) wrote me the following:

Also, I tried to quickly recreate the chain of events that lead to the error, and here's what I noticed:
1. On 03.06 you disabled the job in the middle of merge:
[03.06.2020 17:51:50] <01> Error Transform step failed
[03.06.2020 17:51:50] <01> Error Job has been disabled by user VEEAM\admin (Veeam.Backup.Common.CDisableSessionException)
2. Since then it began to fail with the same error on every run:
[03.06.2020 18:02:26] <01> Error Unable to create a restore point: some backup files backing previous restore points are offloaded, please run an active full backup (System.Exception)
3. Until you forced active full on 05.06, after which it failed with the DB error:
[05.06.2020 17:26:13] <01> Info Violation of UNIQUE KEY constraint 'UQ_CryptoKeyRecoveryRecs_EncryptedKeyId_RecoveryKeyId'. Cannot insert duplicate key in object 'dbo.CryptoKeysRecoveryRecs'. The duplicate key value is (221bc60a-56bc-4584-862b-8d30e7a434c4, 83898d5a-41b0-44aa-ae15-d921dd25f8f6).
[05.06.2020 17:26:13] <01> Info The statement has been terminated.]
[05.06.2020 17:26:13] <01> Error Microsoft SQL server hosting the configuration database is currently unavailable. Possible reasons are heavy load, networking issue, server reboot, or hot backup.

So it looks like it was the cancelled merge that caused the error. Moreover, it might be that it caused the other error too.
Now I needed to create a new BCJ. Now there's no need to blame anyone for that, the only thing that always annoys me is the fact that due to a new BCJ, you loose the whole deduplication-benefit on s3 and unfortunately it's not the first time that I had to start with a new chain/BCJ due to issues.

Just wanted to post this case here so that the product can be improved.

Thanks!
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: BCJ / offload destroyed by cancelling BCJ?

Post by veremin »

Can you share the ticket ID with us? Thanks!
mcz
Veeam Legend
Posts: 835
Liked: 172 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Location: Rheintal, Austria
Contact:

Re: BCJ / offload destroyed by cancelling BCJ?

Post by mcz » 1 person likes this post

Vladimir, the ticket # is 04107801, but many other issues in combination with s3 have been handled there if I'm not mistaken. So you might not find what you're looking for if you start reading the case from the beginning ;)
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: BCJ / offload destroyed by cancelling BCJ?

Post by veremin » 1 person likes this post

I've reached QA team. They will verify the described scenario. I will update the topic, once I have more information. Thanks!
mcz
Veeam Legend
Posts: 835
Liked: 172 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Location: Rheintal, Austria
Contact:

Re: BCJ / offload destroyed by cancelling BCJ?

Post by mcz »

Thanks Vladimir. Just another thought: I know that veeam warns you when you abort a BCJ saying that it might lead to data corruption. But when I confirmed, I would have only expected that the currently merged/created restore point could get corrupted so that you would rebuild it in the worst case - or better said veeam would do that for you. I never expected issues like that the whole BCJ could get unusable, which is a bit more severe IMHO. So if that issue was "expected", please adopt the warnings - thanks.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: BCJ / offload destroyed by cancelling BCJ?

Post by veremin »

Honestly, the issue does not look expected. So, we will check the issue on our side to see what might be the root cause. Thanks!
Post Reply

Who is online

Users browsing this forum: Ivan239 and 9 guests