AWS Glacier - API calls estimate

[adlloyd79]

Hi,

So my understanding is that for the capacity tier a rough calculation for the expected AWS API calls is 1,000,000 calls per TB of data, and that this is based on data size rather than backup size. I also understand that with immutability enabled we have to factor in a full backup worth of API calls every 10 days. I hope I have got all that correct so far.

So my question is how would I estimate the API calls for moving the data from the capacity tier to the archive tier in Glacier?

Thanks.

[Gostev]

Hi, that is indeed the correct estimation for the Capacity Tier. However, keep in mind that AWS Glacier can only be used for the Archive Tier, where this calculation does not apply. With the Archive Tier, you can pretty much ignore the API costs due to a different format the data is archived in (huge objects).

Now, if you're asking about API calls for reading data from Capacity Tier as a part of the archival process, then it will largely depend on the archiving mode: default forever-incremental or optional "standalone fulls" mode. In the latter case, it will indeed take a full backup worth of API calls, so you may want to ensure you're not archiving weekly backups (but only monthly/early). While in the former case, much less and depending on a change rate in your environment.

Thanks!

[adlloyd79]

Hi Gostev,

Thanks for coming back to me so quickly. Yes I should have been a little clearer, I will be using S3 Standard for the Capacity Tier, and Glacier for the Archive Tier.

So you say I can pretty much ignore API costs for the Archive Tier (good news, one less thing to try and estimate). However I am now a little confused by your second statement regarding API's for reading data from the Capacity Tier for the archival process. What sort of API's should I be expecting here and how would I estimate volume?

Maybe I should add a little more context to my calculations. So the plan is to have SOBR with local storage for Performance Tier, AWS S3 Standard with immutability for Capacity Tier and Glacier with immutability for Archive Tier. If I use an example VM that has a data size of 180 GB for the full backup and 9 GB for the daily incrementals. I am estimating in the Capacity Tier 180,000 API's for a full backup and 9000 API's for the daily incrementals. So over a 30 day period I would expect 180,000 x 3 (1 full every 10 days for immutability) and 9,000 x 27 = 783,000 API's. At the S3 Standard API cost of $0.0053 per 1000 API's I would estimate $4.15 API ongoing cost for that VM over a 30 day period in the Capacity Tier. So if I am sending a monthly GFS for that same VM to the Archive Tier how many API's (if any) should I expect to have to pay for?

Essentially what I am trying to do here is pull together some rough costs for storing backups in AWS that I can present back to my board to get approval to move forward with this. I am pretty comfortable estimating the storage costs, but less so estimating the API costs.

Many Thanks.

[Gostev]

Offloading data from the Capacity Tier to the Archive Tier requires reading a delta with the previous restore point already in the Archive Tier, from the Capacity Tier. So the number of GET APIs is impossible to estimate: it will depend solely on the amount of changed blocks between the two restore points. It can be really small one month, and really big another month depending on your workload, OS updates etc. Strictly from budgeting perspective though, I guess you can't go wrong by assuming 100% change rate (meaning the entire restore point will need to be read once a month for archival purposes).

You're right in that API costs estimation presents a huge planning challenge. This is exactly why more and more Veeam customers are choosing object storage providers who do not charge for API costs at all, like Wasabi.