A little background why I’m looking for this information. We are experiencing slowness writing to an on premise S3 appliance. Small writes are directly inserted into the database of the system managing the S3 objects vs writing to the underlying storage. Based on what I’ve been able to observe the Put/Get latency for the database is roughly 3-400ms. I have over 1,000 backups pointed at this SOBR.
I would like to know more about the creation/deletion of the check points for offload jobs to S3 storage.
I’m assuming they are similar size no matter the size of the offloads. It would be nice to know an approximate size but what’s more important to me at this time is how the checkpoint file(s) are written/deleted. Is the data written at the block size (default 1M) of the backup job or some other method? Are they written as a bunch of small files or a few large files?
Dave
-
- Enthusiast
- Posts: 69
- Liked: 6 times
- Joined: Aug 07, 2015 8:45 pm
- Full Name: David Engler
- Contact:
-
- Veeam Legend
- Posts: 418
- Liked: 243 times
- Joined: Apr 11, 2023 1:18 pm
- Full Name: Tyler Jurgens
- Contact:
Re: S3 Offload checkpoint management
The data written to S3 will be the block size of the backup job (default 1M, but can be set as high as 8M). Each block will get written to the S3 as its own object - thus you will have many small objects written to S3.
Tyler Jurgens
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
-
- Enthusiast
- Posts: 69
- Liked: 6 times
- Joined: Aug 07, 2015 8:45 pm
- Full Name: David Engler
- Contact:
Re: S3 Offload checkpoint management
A follow up to my earlier question about the check point file management.
When do the expired objects get deleted? Is this done at the end of each VM that is being offloaded or at the end of the SOBR offload when the Object Clean up task takes place?
Dave
When do the expired objects get deleted? Is this done at the end of each VM that is being offloaded or at the end of the SOBR offload when the Object Clean up task takes place?
Dave
Who is online
Users browsing this forum: Semrush [Bot] and 43 guests