Expected Object Storage Behavior?

[isolated_1]

Hoping someone can confirm if what I am seeing with object storage using the copy feature along with Forever incremental is correct.

Setup: I am using forever incremental going to a SOBR with per-vm backup. The scale out repository is configured for the Copy feature.

Behavior I see:

1. Upon first backup completion, I see a object storage job to upload the VBK.
2. Upon incremental backup completion, I see a object storage job to upload the .VIB.
3. Upon reaching the end of the restore point chain, the incremental gets created and I see a job similarly to upload it to object storage as seen previously. Veeam then synthesizes the earliest VIB file into the VBK on production storage. Upon finishing, I see another object storage job that needs to read the entire VBK file again to upload what it needs. It does not matter if only a single text file was changed as the VBK seems like it needed to be re-read altogether.

It is step three that is causing me to not be able to use object storage. At first I was confused a bit as to why it had to re-read the entire VBK file again but I thought about it and realized that yes, if Veeam synthesizes the VIB to the VBK, that means it changed and so obviously that data for the VBK will need to get uploaded as well. I look at object storage being "dumb storage" in that there is no processing done on that end so unlike with backup copy jobs, the destination has no way to do its own synthetic operation on teh files/data. I am only doing this as a test but if this happens every night where all 500 of our VM's VBK need to be re-read, there is no way it will finish and will crush our backup storage.

Is this the expected behavior?

Also, if I enable object storage with the Copy feature on a SOBR that already have 150TB of data across all VMs, will that immediately trigger Veeam to copy all 150TB of data to object storage?

[Gostev]

No, this is not an expected behavior. We don't need to re-read the entire VBK under any circumstances.

There are also no synthetic full operations in object storage, because there is no file system and so no backup files to start with.

[oleg.feoktistov]

Also, if I enable object storage with the Copy feature on a SOBR that already have 150TB of data across all VMs, will that immediately trigger Veeam to copy all 150TB of data to object storage?

No, it should prompt you if all backups should be copied or only those of last active backup chain, as described here. Thanks!

[isolated_1]

No, this is not an expected behavior. We don't need to re-read the entire VBK under any circumstances.

There are also no synthetic full operations in object storage, because there is no file system and so no backup files to start with.

OK, I'll have to retest again and most likely open a case if I see same behavior. During my initial test, I tried it with 3 very small VMs and saw the behavior I mentioned across all three. Understood about the object storage and no synthetic full operations in the backend. That's kind of what lead me to believe the behavior I saw was expected across a forever incremental. But if you're saying that's not expected, then something is wrong on my end. Thanks.

[isolated_1]

No, it should prompt you if all backups should be copied or only those of last active backup chain, as described here. Thanks!

Hey Oleg,

Understood regarding that option. However, how would this work for Forever Incremental jobs? Obviously I will need to upload the entire chain VBK + VIB files to object storage for it to work so I don't have the luxury of copying only the 'latest' data as all data would be required in a Forever Incremental setup if I want to back up to object storage.

I'm afraid that if I turn on this option in production, it will automatically start uploading all of our data to object storage at one time. Obviously there are throttling I can setup and whatnot but my mind can't get away from thinking that all hell will break loose once I turn on object storage with the Copy function in production where 150TB+ of data is sitting on. Thanks.

[oleg.feoktistov]

Hi Simon,

In any cases it will copy backups with the lowest priority and only if dependent backup infrastructure resources can be allocated.
So, intelligent task scheduler and concurrent task slots setting will guard your backup infrastructure availability and performance.

Cheers,
Oleg