Using object storage as a backup target
Post Reply
bit_inspector
Novice
Posts: 3
Liked: never
Joined: Mar 26, 2020 12:40 pm
Full Name: Thomas
Contact:

AWS S3 Infrequent Access: how to reduce the number of API calls

Post by bit_inspector »

Support ID = 04345915

Hello,

we are using AWS S3 Infrequent Access with the immutable feature to copy/offload backups from a Scale Out Repository.

Unfortunately we did not calculate the costs for PUT, COPY, POST, LIST when estimating the price for AWS S3 and our budget is heavily overdrawn.

We initially started at the end of July and as of today the costs for requests for roughly 35 TB Backups accumulated to:

July:
37,613,594 Requests (Amazon Simple Storage Service EUC1-Requests-SIA-Tier1) = 376,14 USD

August:
79,901,319 Requests (Amazon Simple Storage Service EUC1-Requests-SIA-Tier1) = 799,01 USD
84,534,867 Requests (Amazon Simple Storage Service EUC1-Requests-Tier1) = 456,49 USD

- The immutable feature is set to 30 days
- The backups are encrypted
- We run synthetic fulls once a week on ReFS
- Backups older than 8 days are offloaded
- When the repository reaches 80% capacity backups are offload by "override"
- Retention is set to 30 days
- Weekly health checks
- The jobs have the setting "LAN target"

My questions:

- Why are we at 160.000.000 Requests in August, when the Rule of thumb is 1 TB Backups = 1.000.000 Requests? Shouldn't it be more in the 35 to 40 Million region (35 TB of Backups)?
- Why are there 84 Million Request on S3 Standard when the Bucket is configured as IA via Veeam?

- How can we significantly reduce the number of request made by Veeam to cut costs? We need to reduce the requests as much as possible.
- Would it help to switch to incremental forever instead of synthetic fulls once a week?
- Would it help to disable encryption in Veeam?
- Would it help to do less frequent health checks, let's say every 2 or 4 weeks instead of weekly?
- Would it help to lower the retention period from 4 to 2 weeks?

Thank you
Thomas

HannesK
Veeam Software
Posts: 6101
Liked: 885 times
Joined: Sep 01, 2014 11:46 am
Location: Austria
Contact:

Re: AWS S3: how to reduce the number of API calls

Post by HannesK »

Hello,
and welcome to the forums.

There are multiple things to take into account.

1) you doubled the API costs by going away from the default settings. Please see FAQ about you LAN target setting. post338749.html#p338749
2) Immutability costs around one full backup in API costs every 10 days

If you really want to reduce costs, then I recommend going to a cloud provider, that has a user friendly pricing model :-)

Question 2-4: no, that does not change anything from a cost perspective

Question 5: only for disk space. Keep in mind, to also reduce immutability time. (well, if you stay at infrequent access, then there are extra charges for early deletion)

Best regards,
Hannes

bit_inspector
Novice
Posts: 3
Liked: never
Joined: Mar 26, 2020 12:40 pm
Full Name: Thomas
Contact:

Re: AWS S3: how to reduce the number of API calls

Post by bit_inspector »

Hi Hannes,

thank you for your reply.

So "Local Target" should be set instead of "LAN Target", do I understand that right?

But I still don't understand why Standard S3 API costs are charged, when the bucket is set to S3 IA via Veeam?

And how would you explain the total number of requests, 160 million. Even if I set "Local Target", it still would be 80 million request instead of the expected 30 to 40 million (1 TB = 1 million requests)?

Q5: OK, so switching to Standard S3 would cut the API cost bei 50% per the AWS price list. The storage costs will be higher, but in sum we would save money. Regarding the immutability time, what is a recommended time frame by Veeam? I think one must consider, that attacks are sophisticated today, and therefor a couple of weeks are not a bad idea (plus deploying a good IDS/IPS).

Thank you
Thomas

HannesK
Veeam Software
Posts: 6101
Liked: 885 times
Joined: Sep 01, 2014 11:46 am
Location: Austria
Contact:

Re: AWS S3: how to reduce the number of API calls

Post by HannesK »

Hello,
actually I have no idea about the Standard S3 API costs for S3 IA. I don't run that setup.

Yes, one MByte source data = 1 PUT request with Local target (see FAQ). Going to larger blocks would reduce API costs, but increase disk space costs. The defaults are fine form a cost perspective.

Don't forget about the API costs for a full backup every 10 days for immutability. Whether you configure 3 days or 30 days... every 10 days we need to make sure, that all relevant blocks are still immutable. This operation is already cost optimized. The alternative would be to use more disk space (=more costs). If you have 30 days retention, then I would protect around half of it, maybe 14 days. But that's just personal preference.

Keep in mind, that we protect the configured days + 10 to avoid API costs (the user guide explains it some kind of, but I would not bother too much about the details).

Best regards,
Hannes

Gostev
SVP, Product Management
Posts: 26699
Liked: 4274 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: AWS S3: how to reduce the number of API calls

Post by Gostev » 1 person likes this post

Please note that S3 IA is specifically recommended for offloading GFS fulls only (in other words, using the Move policy with a large window). There's a note about this right in the UI under the corresponding check box. Using S3 IA with the Copy policy, which processes every single incremental backup, and with immutability on top will definitely be a huge money drain due to much higher API costs for this S3 tier.

As you can see from your numbers, even with the default block size setting of 1MB (twice larger than your current one), your API costs will still exceed storage costs.

The only way to make S3 IA API costs somewhat reasonable with the Copy policy and immutability enabled is to use the largest 4MB block size in Veeam. However, it will increase your storage consumption due to much larger increments, and also make granular restores directly from object storage slower and more expensive.

All in all, using regular S3 and the default block size usually provides the best balance for most use cases.

Please keep in mind that switching the block size requires an active full backup to be performed.

MrSpock
Service Provider
Posts: 40
Liked: 2 times
Joined: Apr 24, 2009 10:16 pm
Contact:

Re: AWS S3: how to reduce the number of API calls

Post by MrSpock »

Hi,

The major part of our AWS S3 costs does also come from API calls. Take a look at our usage for the last month. The purple parts are API calls. These consist of 31 % PutObject operations and 69 % WriteObjectLockRetentionInfo operations. The amount of PutObject is about the same every day, but WriteObjectLockRetentionInfo is peaking on a few days as you can see in the chart.

I have 3 restore points that are COPIED to AWS S3 with standard storage class and immutability for 5 days.

Is there a way to decrease the amount of WriteObjectLockRetentionInfo operations during the peaking days or is there a way to perform these operations less often?

Image

Thanks,

Johan

HannesK
Veeam Software
Posts: 6101
Liked: 885 times
Joined: Sep 01, 2014 11:46 am
Location: Austria
Contact:

Re: AWS S3 Infrequent Access: how to reduce the number of API calls

Post by HannesK »

Hello,
yes SOBRArchiveImmutabilityGenerationDays (0-30 is possible. 10 is default which I mentioned earlier) does that. But that will increase storage costs. So I'm not sure, whether you really want to change that value...

Best regards,
Hannes

MrSpock
Service Provider
Posts: 40
Liked: 2 times
Joined: Apr 24, 2009 10:16 pm
Contact:

Re: AWS S3 Infrequent Access: how to reduce the number of API calls

Post by MrSpock »

HannesK,

Does that mean that retention information will be changed on all objects each 0-30 days instead of each 10 days? Maybe I should experiment a bit with that value as API costs are much higher than storage costs today.

I found no information about SOBRArchiveImmutabilityGenerationDays in the forums or anywhere else. Is it just a DWORD value in the "Veeam Backup and Replication" registry section on the backup server?

Thanks,

Johan

Gostev
SVP, Product Management
Posts: 26699
Liked: 4274 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: AWS S3 Infrequent Access: how to reduce the number of API calls

Post by Gostev » 1 person likes this post

Yes, all registry values go there.

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests