Backup Copy Jobs to object fail

[tdubbs810]

Hello,

We have a case regarding this open already: Support ID: #07375946 and that is a continuation from this case: Case #07360228

[Moderator: removed Support ID and added correct Case ID]

We have been experiencing a strange, randomly recurring error in some of our backup copy jobs:
Failed to process Error: Cannot proceed with the job: existing backup meta file '{vm.vmb}' on repository '{direct to object repo}' is not synchronized with the DB. To resolve this, run repository rescan.

It usually pops up for one VM at a time, but occasionally affects two or three. It also seems to occur randomly - there were 8 days straight of no errors and then two backup copy jobs started throwing it. The object repo is in Wasabi.

We had a workaround that seemed to fix things:
1. Create a new temporary Backup Job of same type (Not Backup Copy even if the backup copy is failing) with a random VM and encryption enabled using the same password which for the backup copy job that is failing.
2. Go to Backups > Disk > find the VM that is failing > right click > Move > select the newly created backup job.
3. Wait for the Backup Move operation to complete
4. Run the Backup and Backup Copy job again
5. Delete the new temporary job.

This no longer is working on the most recently affected repositories. Cloning the backup copy job and running that succeeded. We have not been able to find any common reason or configuration that could be causing this to happen. While the cloned jobs fix it now, but we're concerned it could keep happening on the cloned jobs at some point too.

[david.domask]

Hi Tyler, welcome to the forums.

Sorry to hear about the challenges here, and thank you for sharing the case number.

Quick sanity check, the buckets backing the Object Storage repositories are used with a single repository exclusively, correct? (i.e., if you have Bucket A, it is only used for a single repository, not multiple repositories?)

Please keep working with Support on this one, as the behavior is not expected and a review of the logs will tell us the answer; my questions above are simple sanity checks as sometimes this behavior can present in such configurations.

If you have concerns regarding the processing of the case, please use the Talk to a Manager button from the Case Portal; Support Management will review the case and help to ensure proper resources are aligned for the case.

[tdubbs810]

Hi David,

Correct, each bucket is 1:1 to a repo. We just received an escalation on our support case, so I am hoping we can get more answers soon.