Understanding the Backup Cache

[BackupBytesTim]

From reading this: https://helpcenter.veeam.com/docs/agent ... tml?ver=60

It seems the backup cache behaves about as I deduced on my own, my current understanding is that if a backup gets created in the cache, the next time a backup job runs, that backup will also get placed in the cache and then after the backup job the agent may attempt to upload the entire contents of the backup cache, even if the target repository was available at the time the job started.

Is that accurate or would the agent attempt to upload the contents of the cache before running the new backup job?
Do I understand correctly that if the cache is completely empty and the target repository is available, the agent will backup directly to the repository and the cache is not used?

I ask mostly because we frequently encounter issues where backups have ceased altogether on account of the cache being full, my understanding is that if the cache can not accommodate the next backup then it fails completely because it will not try to upload, and then erase, the contents of the cache until the next backup completes. Currently we resolve this issue by disabling the cache entirely, restarting the agent, and running the job again, which has the desired affect of deleting the contents of the backup cache. Deleting the cache is not necessarily desirable though, if there's a way to manually initiate a cache upload job, that would be beneficial. However, deleting the cache and losing whatever was in it is preferable to having no future backups occur, which seems to be what happens now.

I'm considering that we're better off just disabling the cache feature entirely since the cache is always located on the same source drive the desired data to be backed up comes from, so in the event of a hardware failure or ransomware encryption, the data is lost anyways.

Also, I am correct in assuming if the cache location is included in the backup source selection, it will not be included, right? For us this typically means it's on a volume included in the Entire Computer setting, but I'd also be curious to know if the exclusion behavior exists if the folder is included either explicitly or as a child of a folder specified in a Files backup.

[Mildur]

Hi Tim

Is that accurate or would the agent attempt to upload the contents of the cache before running the new backup job?

The restore point in the cache gets uploaded as soon the remote repository is available again. Our agent starts a backup cache synchronization job in the background if there is at least one restore point in the cache. That job monitors the availability of the remote repository. When it becomes available, the restore points gets moved from the cache to the target repository.
If a new job starts and there are still restore points in the cache, the job will create a restore point in the cache folder instead of the target repository.

Do I understand correctly that if the cache is completely empty and the target repository is available, the agent will backup directly to the repository and the cache is not used?

Correct. If there is no previous restore point in the cache and the target repository is available, the restore point will be saved to the target repository. Cache won't be used.

I ask mostly because we frequently encounter issues where backups have ceased altogether on account of the cache being full, my understanding is that if the cache can not accommodate the next backup then it fails completely because it will not try to upload, and then erase, the contents of the cache until the next backup completes.

If you use Caching, enough space must be available to store any restore points created. A manual button is not required. I tested the scenario and my restore points were uploaded within 1-2 minutes by the background synchronization job. I suggest to open a support case if that doesn't happen for your clients.

Also, I am correct in assuming if the cache location is included in the backup source selection, it will not be included, right? For us this typically means it's on a volume included in the Entire Computer setting, but I'd also be curious to know if the exclusion behavior exists if the folder is included either explicitly or as a child of a folder specified in a Files backup.

Correct, we add the cache folder path to the following registry key before creating the vss snapshot:

Code: Select all

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshot

Best,
Fabian

[BackupBytesTim]

Thanks for the information, that clarifies everything.

Next time I get a job that's stuck because the cache is full and it's not getting uploaded I'll open a case to troubleshoot it further.

[BackupBytesTim]

So it's been a while and there have been more issues, but right now I have a bunch (like 20) computers failing to upload their cache. Some show running status in VSPC, some don't, some show errors, some don't (on the VSPC), and some show data transfer in the VCC console, and some don't.

There's really no consistency. But there's a lot of things that are failing to backup because the cache isn't working and so new backups won't work either. I've had problems with the cache for basically ever, but this many "broken" backup jobs all at once has me wanting to just take the time to go through every single computer and disable the cache every where, it seems to cause far too many problems.

But before I do that, I thought I'd try one last (last, because it's the third and I'm tired of continuing to try and fix something when I could just stop using it, referring to the cache) post here first, especially since this particular one says "the cache is full" between running.

So the computer shows the running job in the VCC console, a backup cache sync job, and it shows it's transferring data. And looking at the files (yes, yes, I know I'm not supposed to, but it's helpful for troubleshooting, like in this scenario) there is a VIB file that gets modified while it's running. But the file size never changes, it got up to 15.9 GB and just has been the same for several days now. The job doesn't run for very long at a time before it stops, when it's not actively running the VSPC shows an error message about the cache being full. From what I've learned, I believe the error is pertaining to the primary backup job, it can't back up to the cache because the cache is full. So I'm assuming that has nothing to do with the actual cache failing to sync to the VCC repository.

But I could be wrong, the VSPC doesn't really give much information.

Support case: #06392254

Most (but not all) of the computers that have had issues lately have been resolved by simply disabling the cache, restarting the agent, and running a backup. I'm hoping to figure out what the issue is though, not just apply a quick fix.

This did happen across numerous different computers at about the same time, so I'm assuming the issue was caused by something on the VCC infrastructure, but I can't think of anything that would have caused an issue that can be fixed by disabling the cache on the agent, so I have no ideas for what caused the issue in the first place or what might could be done to fix it other than disabling the cache.