Multipart Upload Behavior for S3 Backup Copy Jobs?

[rin]

Hi,

While reviewing the behavior of backup copy jobs to S3-compatible object storage, we noticed that there is no explicit documentation or configurable option describing how multipart upload is handled.

To better understand the product behavior from a design perspective, we would like to confirm the following points:

1. When Veeam Backup & Replication uploads backup data to S3-compatible object storage, does it internally use S3 Multipart Upload based on object size or other conditions?
2. Is this behavior fully automatic and not configurable by users (that is, no supported option to enable, disable, or limit multipart upload usage)?
3. Are there any official considerations or recommendations regarding incomplete multipart uploads (for example, cleanup or lifecycle handling), or is this entirely managed by the object storage side?

We would appreciate your clarification to ensure an accurate understanding of the product behavior.

Thanks,
Rin

[david.domask]

Hi Rin,

Veeam uses a unique structure for backups to Object Storage repositories; the data blocks from the backup file are "chunked out" into smaller objects and sent to S3 instead of sending the entire backup file as a single entity.

In short, there is no need for end-users to adjust anything regarding MultiUpload.

3. Are there any official considerations or recommendations regarding incomplete multipart uploads (for example, cleanup or lifecycle handling), or is this entirely managed by the object storage side?

Neither, it is all managed by Veeam. It is imperative that you do not enable life-cycle policies or manually alter data on the bucket, the data must be managed by Veeam exclusively.

[rin]

Hi Domask,

Thank you very much for your detailed explanation.

This clarified the internal behavior and helped me better understand how Veeam manages data in object storage. The explanation about the chunked structure and lifecycle handling was especially helpful.

I appreciate your support.

Thanks,
Rin

[rin]

Hi Domask,

Thank you again for your previous clarification — it was very helpful and gave me a much clearer understanding of how Veeam manages data in object storage.
After reviewing the explanation in more detail, I realized I would like to confirm a few additional points to make sure I fully understand the behavior correctly.

1. Based on your explanation, my understanding is that Veeam does not rely on the traditional S3 Multipart Upload concept where a single file is split into parts and uploaded as one logical object.
Could you please confirm if this interpretation is correct?

2. If so, could you clarify how the data is actually uploaded to S3-compatible object storage?
Specifically:
・Is the data pre-chunked into multiple independent objects before upload?
・Does Veeam primarily use standard PUT object operations for these chunks, or are any S3-specific mechanisms involved internally?

I apologize for the follow-up, but I would like to ensure I fully understand the design correctly.

Thanks,
Rin

[david.domask]

Hi Rin,

No worries, questions are what the forums are for.

1. Multi-upload is used in parts of the process (that is, if you check your API call stats on your S3 system you may see some multi-part upload calls), but mostly you will just see PUT Object calls

2. VBK / VIB files are best understood as containers for the backup data; all the workload data is stored as a File in Backup (FIB), and it's stored in blocks of size = storage optimization settings. What you set here for the job will be the size of objects uploaded to S3. Our recommendation for most jobs continues to be our default; some S3 vendors advise using bigger block sizes to reduce API calls, which will work, but this has the caveat that over time the storage consumption on S3 may be higher depending on the workload. Again, our default is recommended.