Discussions related to using object storage as a backup target.
Post Reply
redfasdfasf
Novice
Posts: 7
Liked: never
Joined: Mar 05, 2021 10:12 am
Contact:

Estimated number of S3 objects

Post by redfasdfasf »

Hi everyone,

although this has been discussed before, I am not able to find a definite guideline to calculate or estimate the number of objects that my backup will produce on S3.
As my object storage provider (not AWS) is having a limit on the number of objects per bucket, I am planning to redesign my current setup.
To use a SOBR, I want to provide a good amount of buckets for the capacity tier before writing data to balance data across buckets and meet with the cloud provider requirements now and for the foreseeable future.
I want to limit each bucket in size using the bucket properties.
I did a few quick tests, but I am not sure if this can be used as a rule of thumb. Result was that the same for SOBR targets, Backup to S3 directly, and with object lock and enabled or without.
Result was:
Data processed: 107GB
Data read: 55,3 GB
Data transferred: 37,6 GB.
Block size is 1 MB, number of object in S3 are 55400.
Can I assume that that the data read in GB *1024*1024/block size is the amount of objects?
In that case I would limit my buckets to approx. 50 TB when 50 million objects are supported and 1 MB block size is used?

Thanks
sfirmes
Veeam Software
Posts: 321
Liked: 150 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Estimated number of S3 objects

Post by sfirmes »

We do have Veeam Calculators that can help you with your sizing estimates. Also, if your object storage provider can give you a size limit on their buckets you can use VBR 12.3's Automatic bucket provisioning capability to automatically generate "sub-buckets" for stand-alone or performance tier repositories.
Steve Firmes | Senior Solutions Architect, Product Management - Alliances @ Veeam Software
redfasdfasf
Novice
Posts: 7
Liked: never
Joined: Mar 05, 2021 10:12 am
Contact:

Re: Estimated number of S3 objects

Post by redfasdfasf »

Hi,
thanks for the reply. I looked at the Calculator and I haven't see a way to calculate the number of objects on the object storage.
Also, multi-bucket does not work on a scale-out with object lock enabled and the policy for new buckets only provides number of workloads/ backup chains, not the actual size.
Post Reply

Who is online

Users browsing this forum: No registered users and 14 guests