-
- Novice
- Posts: 3
- Liked: never
- Joined: Jan 31, 2019 10:00 am
- Full Name: Sebastian Kuliś
- Contact:
S3 requests calculator
Many S3 calcs and pricings include, beside of storage price a PUT/COPY/POST/LIST Requests and GET/SELECT and Other Requests. I am assuming that these are not major costs but I want to predict all costs.
Whats these values are or how to calculate them per 1 job i.e.: backup job or restore vbk from cloud to on-premise.
Whats these values are or how to calculate them per 1 job i.e.: backup job or restore vbk from cloud to on-premise.
-
- Chief Product Officer
- Posts: 31606
- Liked: 7097 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3 requests calculator
It does became largely unnoticeable if you do long term retention in object storage. However, if you are only storing data for a few months, it can in fact become a very noticeable part of your bill.
Number of API calls can roughly be estimated as the number of non-zeroed 1MB blocks in source images. For example, if your VMs take up 1TB of disk space total, then initial offload will require 1 million API calls. From that point on, it's incremental forever so is dependent on the change rate.
Number of API calls can roughly be estimated as the number of non-zeroed 1MB blocks in source images. For example, if your VMs take up 1TB of disk space total, then initial offload will require 1 million API calls. From that point on, it's incremental forever so is dependent on the change rate.
-
- Novice
- Posts: 3
- Liked: never
- Joined: Jan 31, 2019 10:00 am
- Full Name: Sebastian Kuliś
- Contact:
Re: S3 requests calculator
In my scenario plan is BCJ with GFS (monthly, yearly) and only Full BCJ offload (increments only locally). Initial and so monthly 16TB of data, so I should calculate 16 million API calls per month?
-
- Chief Product Officer
- Posts: 31606
- Liked: 7097 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3 requests calculator
Correct for the first month, and much less in the following months, because the following GFS fulls will be offloaded incrementally (only delta with the previously offloaded fulls).
-
- Certified Trainer
- Posts: 1024
- Liked: 448 times
- Joined: Jul 23, 2012 8:16 am
- Full Name: Preben Berg
- Contact:
Re: S3 requests calculator
It is much more than that. I literally just looked at our load balancer, and the average block size is average 300-400 KB. To me, it looks like the VBK/VIB file is simply "exploded" directly into object storage without any optimizations. We have worked with 3-4 object storage vendors who are claiming that Veeam is offloading only 1 MB blocks, but this is a false statement.
We only use Veeam in "Copy Mode". Does it work differently than for "Move Mode" in terms of block size?
-
- Veteran
- Posts: 405
- Liked: 106 times
- Joined: Jan 30, 2017 9:23 am
- Full Name: Ed Gummett
- Location: Manchester, United Kingdom
- Contact:
Re: S3 requests calculator
Hi @poulpreben, the 1 MB (default) is size of source blocks, so before compression. 2.5-3.3x compression would result in the sizes you’re seeing.
Copy and Move work in the same way, the difference is just what recovery points are uploaded.
Object storage utilisation can be calculated at https://vse.veeambp.com/ and for background on the logic please see https://www.veeam.com/wp-designing-budg ... -tier.html
Copy and Move work in the same way, the difference is just what recovery points are uploaded.
Object storage utilisation can be calculated at https://vse.veeambp.com/ and for background on the logic please see https://www.veeam.com/wp-designing-budg ... -tier.html
Ed Gummett (VMCA)
Senior Specialist Solutions Architect, Storage Technologies, AWS
(Senior Systems Engineer, Veeam Software, 2018-2021)
Senior Specialist Solutions Architect, Storage Technologies, AWS
(Senior Systems Engineer, Veeam Software, 2018-2021)
-
- Chief Product Officer
- Posts: 31606
- Liked: 7097 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3 requests calculator
Right... he even quoted that himself
-
- Certified Trainer
- Posts: 1024
- Liked: 448 times
- Joined: Jul 23, 2012 8:16 am
- Full Name: Preben Berg
- Contact:
Re: S3 requests calculator
Oh, I see. I misunderstood “non-zeroed” blocks. Since it was in the context of source storage, I thought you were referring to thin provisioning.
The VSE link is 404. Is there a calculator there to “roughly” estimate the amount of blocks?
The VSE link is 404. Is there a calculator there to “roughly” estimate the amount of blocks?
-
- Veteran
- Posts: 405
- Liked: 106 times
- Joined: Jan 30, 2017 9:23 am
- Full Name: Ed Gummett
- Location: Manchester, United Kingdom
- Contact:
Re: S3 requests calculator
Sorry, proofreading fail!
https://vse.veeambp.com/
transaction costs are shown, but counts can be downloaded if you log in - DM me if you want login details
https://vse.veeambp.com/
transaction costs are shown, but counts can be downloaded if you log in - DM me if you want login details
Ed Gummett (VMCA)
Senior Specialist Solutions Architect, Storage Technologies, AWS
(Senior Systems Engineer, Veeam Software, 2018-2021)
Senior Specialist Solutions Architect, Storage Technologies, AWS
(Senior Systems Engineer, Veeam Software, 2018-2021)
-
- Chief Product Officer
- Posts: 31606
- Liked: 7097 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3 requests calculator
Probably no need for a calculator for rough estimation: 1TB disk = 1 million blocks = 1 million API calls. You may have less API calls due to zeroed and other repeating blocks, but never more.poulpreben wrote: ↑Sep 22, 2020 4:30 amIs there a calculator there to “roughly” estimate the amount of blocks?
Who is online
Users browsing this forum: No registered users and 6 guests