Azure Storage Cost Increase for Immutability

[PXAbstraction]

Hey everyone.

I know this topic has come up before but in doing searches, I haven't found an exact answer so I'm hoping someone can help out. We have a client we manage Veeam backups for who is rather cost sensitive. They currently do backups to on-prem storage and also do Backup Copy jobs of those backups to an Azure blob with object storage and a 7 day retention period. Due to recent insurance changes, they're required to implement immutability. Initially, they'd like to just have us create a new storage account and start sending those 7 days of backup copies with immutability enabled. In terms of raw storage, this shouldn't take much more space than they currently use, save the extra chunk for Block Generation. However, I've seen several people say their Azure storage costs have increased substantially due to the number of Put operations that immutability requires, even when using a Hot storage tier.

Does anyone have experience with this who might be able to offer some insights? We can't have our client get surprised with a sudden massive storage increase, but they're also being required to implement immutability so if that big increase is unavoidable, I can at least present them the option before implementation.

Thanks all.

[edh]

Simple , move to other storage provider that not charge for API operations.

Azure is a Ferrari.

[PXAbstraction]

My employer is a Microsoft partner. It's not up to me. Also, that doesn't answer my question.

[Gostev]

But you're asking how to make a Ferrari cost less... it's a hard question to answer

Some recommendations:
- Use the hot storage class, don't bother with anything else (modeling has been done)
- With the hot storage class and Azure, most added expense will be from keeping data longer than usual, so the shorter the immutability period is, the lower added costs you will be. The minimum immutability period you can set is 7 days.

[PXAbstraction]

I get that Azure is a Ferrari, it's just what I'm stuck with so I guess you could say I'm trying to figure out how to get a Ferrari run on 87 octane instead of 93. So the Hot storage tier was planned from the get go because this is for short-term backups. They might want to add longer term GFS stuff down the line, but I'll worry about that then. 7 day retention is actually what they want and they are already aware that Veeam will extend this longer by a certain degree for Block Generation. Based on the modeling you referenced, do you think we could expect a significantly larger storage cost as a result of API operations for this, versus what they have now from having the same backups writing to Azure in a non-immutable fashion? Thanks!

[Gostev]

API costs for immutability extensions are very low with Azure actually, so with such a low retention policy I expect most of added cost to be from extra storage consumption. Up to 3x more as a whole system is optimized for more typical retention policy settings e.g. 30 - 90 days vs. 7 days.

[PXAbstraction]

So just so I'm clear, their costs could still substantially increase because of the lower retention? I know you're saying "up to 3x more", but is that a likely scenario if most of their servers aren't changing substantially between backup cycles? Budgeting this accurately has been a real headache as the publicly available documentation really doesn't go into detail about potential increases in storage usage, beyond talking about the 10 day Block Generation buffer, which doesn't have any indication of 3x more cost. I'm just trying to be as clear on this as I can be because it looks like we're going to have to go to our client with new cost estimates for an already approved project and if we have to do that, I hope to only have to do it once. Thanks for your help.

[Gostev]

Correct. If they used a more typical, longer retention policy, then cost increase from enabling immutability would have been much lower. But with such a low retention policy, the cost increase is much more articulated.

[PXAbstraction]

So, would you suggest something more like 30 days as a better option for mitigating the cost increases? I'm going to have to go back to them with options and ask which increase they can tolerate more. They're being mandated by insurance to enable immutability so they're going to have to make a choice one way or the other. I don't suppose one of the Veeam online calculators would have us figure out the math per chance? Thanks!

[Gostev]

Well, generally speaking 7 days is pretty meaningless as far as offsite retention policies go because it simply won't give customers enough time to notice and react to a data loss or a cyberattack in many cases. So yes, a longer retention policy is highly recommended: at an absolute bare minimum I would never do less than a month (min 14 daily + 4 weekly), and ideally I would try to cover a quarter with 30 daily and 12 weekly.

[PXAbstraction]

No disagreement there, I'm just working with some frustrating limitations. This is a municipal government, so it sounds like they just want this minimum until their next fiscal year, in which case they hope to get a more comprehensive plan approved. I'll try to run things through Veeam's calculator and see if I can get a rough idea.

[PXAbstraction]

So, I checked the client's current Azure storage account and there's approximately 13.5TB in it (rounded up). I ran that through the simple storage calculator with 7 days of immutable retention. This is what I got from it.



If I up this to 14 days, it only adds about 2.4TB to the storage and the number of PUT operations basically doesn't change. The client is currently using almost 4.14M PUT operations per month so this will add about 2M to that, but that's not a massive cost increase. This doesn't appear to add the 3-4x cost you mentioned earlier, but I've already missed some things coming into this so there may be more missing now.

Does the Veeam calculator take into account the extra storage/retention required for Block Generation? I assume it must. If so, do these numbers seem more or less right to you? If so, I think we can sell our client on this pretty easily.

Thanks for your help, I appreciate it.

[Gostev]

No, Veeam calculator does not currently account for object storage immutability peculiarities.

[PXAbstraction]

So, the documentation doesn't lay it out and the calculator doesn't take it into account either? I don't know how people are supposed to accurately budget this. At this point, am I correct in assuming there's no real way to determine their costs except to guess and hope that's right?

[Gostev]

Well, the documentation definitely lays it out quite thoroughly.

I cannot comment on the calculator b/c it's not a product of Veeam R&D, I just know it does not account for object storage immutability peculiarities at this time.