S3 requests calculator

[Sebastian Kulis]

Many S3 calcs and pricings include, beside of storage price a PUT/COPY/POST/LIST Requests and GET/SELECT and Other Requests. I am assuming that these are not major costs but I want to predict all costs.
Whats these values are or how to calculate them per 1 job i.e.: backup job or restore vbk from cloud to on-premise.

[Gostev]

It does became largely unnoticeable if you do long term retention in object storage. However, if you are only storing data for a few months, it can in fact become a very noticeable part of your bill.

Number of API calls can roughly be estimated as the number of non-zeroed 1MB blocks in source images. For example, if your VMs take up 1TB of disk space total, then initial offload will require 1 million API calls. From that point on, it's incremental forever so is dependent on the change rate.

[Sebastian Kulis]

In my scenario plan is BCJ with GFS (monthly, yearly) and only Full BCJ offload (increments only locally). Initial and so monthly 16TB of data, so I should calculate 16 million API calls per month?

[Gostev]

Correct for the first month, and much less in the following months, because the following GFS fulls will be offloaded incrementally (only delta with the previously offloaded fulls).

[poulpreben]

Number of API calls can roughly be estimated as the number of non-zeroed 1MB blocks in source images.

It is much more than that. I literally just looked at our load balancer, and the average block size is average 300-400 KB. To me, it looks like the VBK/VIB file is simply "exploded" directly into object storage without any optimizations. We have worked with 3-4 object storage vendors who are claiming that Veeam is offloading only 1 MB blocks, but this is a false statement.

We only use Veeam in "Copy Mode". Does it work differently than for "Move Mode" in terms of block size?

[gummett]

Hi @poulpreben, the 1 MB (default) is size of source blocks, so before compression. 2.5-3.3x compression would result in the sizes you’re seeing.
Copy and Move work in the same way, the difference is just what recovery points are uploaded.
Object storage utilisation can be calculated at https://vse.veeambp.com/ and for background on the logic please see https://www.veeam.com/wp-designing-budg ... -tier.html

[Gostev]

Right... he even quoted that himself

[poulpreben]

Oh, I see. I misunderstood “non-zeroed” blocks. Since it was in the context of source storage, I thought you were referring to thin provisioning.

The VSE link is 404. Is there a calculator there to “roughly” estimate the amount of blocks?

[gummett]

Sorry, proofreading fail!
https://vse.veeambp.com/

transaction costs are shown, but counts can be downloaded if you log in - DM me if you want login details

[Gostev]

Is there a calculator there to “roughly” estimate the amount of blocks?

Probably no need for a calculator for rough estimation: 1TB disk = 1 million blocks = 1 million API calls. You may have less API calls due to zeroed and other repeating blocks, but never more.