9.5 Update 4 and Microsoft Azure Blob Storage

[veremin]

Correct, hot and cold tiers are supported, archival - not. Thanks!

[Gostev]

This of course changes pricing by x5. Is a VLT on Azure the only solution for this for now then? Even with 30 TB, the difference is significant.

Actually, as usual in this life, it's really not as straightforward and easy as just looking at the price... archive storage has its own peculiarities which very well balance its lower cost.

For example, with regular object storage today, we're able to offload backups in forever-incremental manner, effectively providing source-side dedupe between multiple full backups. Also, we're able to perform block-level retention to immediately and granularly remove any unused blocks. This is possible because we're able to update metadata sitting in object storage in real time.

None of this will be possible with archive storage due to its nature, so we'd have to resort to classic incremental chains with periodic fulls instead of forever-incremental to make this "special" object storage work while keeping retention processing possible. Much like you will have to do it with your VTL workaround, anyway. However, as soon as you have created just a few periodic fulls, you suddenly find yourself consuming 5x more object storage than Capacity Tier would, which obviously kills all those pricing benefits.

And note that we did not even get to the time and the cost of performing restores from archive storage, which is even a sadder story.

[andriktr]

Do I understand correctly that with U4 update we can put backup copy jobs directly to the Azure Blobs ?
Thank you.

[Gostev]

No, your understanding is not correct: backing up (or copying backups) directly to object storage is not supported.

[dimaslan]

Actually, as usual in this life, it's really not as straightforward and easy as just looking at the price... archive storage has its own peculiarities which very well balance its lower cost.

For example, with regular object storage today, we're able to offload backups in forever-incremental manner, effectively providing source-side dedupe between multiple full backups. Also, we're able to perform block-level retention to immediately and granularly remove any unused blocks. This is possible because we're able to update metadata sitting in object storage in real time.

None of this will be possible with archive storage due to its nature, so we'd have to resort to classic incremental chains with periodic fulls instead of forever-incremental to make this "special" object storage work while keeping retention processing possible. Much like you will have to do it with your VTL workaround, anyway. However, as soon as you have created just a few periodic fulls, you suddenly find yourself consuming 5x more object storage than Capacity Tier would, which obviously kills all those pricing benefits.

And note that we did not even get to the time and the cost of performing restores from archive storage, which is even a sadder story.

Gostev, I understand the benefit you are describing as well as the drawbacks of Archive tier. It is, however, ideal for situations where you need to store monthly and yearly full backup files which will only in a case of audit or emergency need to be read. Especially as an MSP, even if you need to do this for 3-4 customers, it might mean adding 10+ TBs per month new storage space to accomodate the new monthlies. With 0.0019 $ / GB, even 50 TBs amount to roughly $ 100 / month for the space which is great, considering the fact it saves the fuss of managing + maintaining storage and the headache of expanding the volumes.

[Gostev]

Dimitris, perhaps you misunderstood my point then.

What I am saying is that in the scenario when you need to retain multiple monthly and yearly full backups, Capacity Tier on cool blobs will be cheaper than storing standalone fulls on archive blobs. While archive blobs are indeed 5 times cheaper, you will need 10 times more storage to store 12 monthly and a few yearly standalone full backups there. So overall, when using archive blobs you will pay more - not less.

[dimaslan]

Gostev,

I will need to read a little bit more about how the tiers work, then.

Thank you.

[Gostev]

Here's the relevant snippet from the What's New document:

Space efficiency: In addition to source-side compression, backup files
are offloaded to object storage in the forever incremental manner, thus
preventing duplication between multiple full backups, which effectively
provides source-side deduplication

[PGoldi]

Hello,
is it now possible to select the Azure Blob Storage as target for a backup copy job?

best wishes
Patrick

[wishr]

Hi Patrick,

Welcome to Veeam Community Forums and thanks for posting your question.

I've moved your post to an existing discussion around Capacity Tier capabilities introduced with B&R 9.5 U4 release worth checking.

Answering your question, no it is not possible to target blob storage using copy jobs. Object Storage integration is a part of Capacity Tier of Scale-Out Backup Repositories. There is a post above by Gostev explaining the purpose of this feature.

Thanks and let us know if any additional questions arise,
Fedor

[brekus]

Hello all,

Quick question about the capacity tier going out to Azure. When a chain is closed, and that full and incrementals are sent out to azure, if there is already a full out there is Veeam/Azure smart enough to know if some blocks are already out and haven't changed? Out bandwidth out isn't the greatest so I am hoping that it's smart enough to know whats already out there.

Thanks!

[veremin]

Correct, identical blocks won't be sent second time. Thanks!

[Manuji]

Hello Team,
New to Veeam. I have a quick question on Veeam B&R and azure blob storage. If I create a copy on prem and want to duplicate it to the azure blob storage (D2D2C) all from one job. is it possible?
Secondly is Veeam B&R compatible with hot,cool & archival tier in azure?

[veremin]

If I create a copy on prem and want to duplicate it to the azure blob storage (D2D2C) all from one job. is it possible?

Copy policy is not included in Capacity Tier today, but we're working on it already. For now you can specify a certain period of time (operational restore window), after which old backups belonging to sealed backup chain will be moved to object storage.

Secondly is Veeam B&R compatible with hot,cool & archival tier in azure?

Only hot and cool access tiers are supported.

Thanks!

[Manuji]

So if I understand this if I create a full b/up and I am part SOBR I cannot create copy job of this full b/up to azure blob storage BUT as of now I can only specify when this full b/u is say 20 day old move it to Azure storage. Correct?

[Gostev]

Correct. Think about it as a way to extend capacity of the particular repository infinitely. Capacity Tier does not give you a different repository, but rather merely extends capacity of an existing one.

[vchris]

It is unlikely that immutable storage will ever be supported for Capacity Tier due to its WORM nature. There are just too many issues around "online storage" use case with most basic stuff that you don't immediately realize, like inability to cleanup incomplete or failed writes even.

However, we will certainly consider immutability for possible future Archive Tier, which is a different use case from Capacity Tier that will likely have vastly different integration approach - due to Glacier-type storage peculiarities and those issues enabling immutability brings. Which also means that will lose most of the benefits of Capacity Tier, and it will no longer be "beautiful" limiting you to true archival scenarios (essentially making it an off-site tape archive replacement).

Hello Gostev, I was trying to use the immutability, time retention not legal hold, mechanisms to avoid the "IT went rogue" on a short interval (beside of physical tapes that stay the better protection into a safe not handled by IT).

It’s a scenario I prepare for a Veeam On Tour.

The setup/objective was to use it on Backup Copy - GFS with 5-7 points, points older than 1 week go to “Capacity Tier”, that way the "recursive" chain stay on the "Perf tier" and the weekly (5 weekly setup) go to the "Capacity Tier" and stay protected from deletion for 4 weeks with immutability “Time retention”.

It does work but we have sometime a cleanup warning (because of immutability immediately activated, I wish we could delay the lock for like one hour after the last modification as some other solutions offer).

Does that usage make any sense to you ?

Of course the futur “Archive Tier” you speak about make more sense, to replace tape out and use WORM like policy, but here the objective is to protect even short term retention outside the Archive Tier.

(On the side I’ve setup StarWinds VTL with Cloud Replication and immutability, and of course it does work well).

[Gostev]

Unfortunately, I am not well aware of technical details Azure implementation of immutability. But I know it's not that simple and straight-forward with Amazon, and none of the above will provide the desired mechanisms to avoid the "IT went rogue". So, it needs to be researched much deeper.

For example, with Amazon object-level immutability is more of a marketing term, in reality what they sell behind this term is regular object versioning. This in turn means, for example, that you still CAN delete immutable objects through S3 API (resulting in an object ending up in a special state with the deleted flag). More importantly though, hackers or malicious IT folks (or ransomware) can still freely overwrite immutable blocks with some rubbish content. And while again, the immutable version of the block will still be preserved "somewhere", restores will fail because they will be using the latest version of the block (with rubbish content).

The good news is that contrary to my earlier response you're quoting, we did seem to find a way to potentially use the object-level immutability feature (aka PUT Object retention) with the Capacity Tier. We're still largely in the design phase though, so it's a bit too early to talk about details. Also, it's currently supported by Amazon S3 only (Azure blob storage only supports container-level immutability that is not fully compatible with our Capacity Tier architecture, as per the warnings you're seeing).

[vchris]

Thank you for the updates

[mcz]

For example, with regular object storage today, we're able to offload backups in forever-incremental manner, effectively providing source-side dedupe between multiple full backups. Also, we're able to perform block-level retention to immediately and granularly remove any unused blocks. This is possible because we're able to update metadata sitting in object storage in real time.

Anton, simple question regarding deduplication on object storage: Is deduplication only performed between the same chain/vm or is it performed across the whole repository? Let's assume that we've got 5 vm's, backed up by 2 different tasks and multiple fulls.

Now I guess that veeam is performing deduplication for at least the same vm in the same chain. But what about other vm's from a different jobs / different chains? Does deduplication across the whole object storage take place or will it only be limited to the same vm/chain?

Thanks for the clarification!

[Gostev]

@mcz deduplication is within the same chain, so the answer depends whether you're using single or per-VM backup files with your primary jobs. Thanks!

[mcz]

Thanks for the answer, Anton. Yes, we're using per-VM backup files, so we're not benefiting that much from deduplication in this case. I assume that things will change when the next update will bring us the functionality to do direct backups & copy jobs to object storage. There I would expect to have the option to switch to "all-in-one" as we can currently using local repos.

Am I right?

[Gostev]

Typically over 90% of deduplication benefits come from dedupe between multiple fulls (periodic and GFS), whereas deduplication across VMs is minimal in the bigger picture. I mean, you can make the numbers looks really good in a POC with freshly deployed VMs, but after just a few month of OS updates VMs will start to deviate so much that you will see very little benefits in the long run. Because of that, I personally would stick to per-VM chains.

[WasabiDan]

It's not really there, look at its size what is left on NAS is a VBK stub containing metadata only - this serves as a local cache, for performance considerations and to reduce the number of API calls which object storage providers often charge extra for.

FYI - Dan from Wasabi.com here - as of this week at VeeamOn we are freshly minted in the "Veeam Ready Program for Object Storage" - and for those who don't know Wasabi yet, not only do we not charge for API calls, we don't charge for egress either, just a flat price per gigabyte of S3 API-compatible storage. We're now 2 years into our publicly available cloud storage, and we heard early on in the market that microcharges were the devil. (or words to that effect )

From an engineering elegance perspective, I love the Veeam approach of a VBK stub as a local cache (and we've found many software solutions that are incredibly... lazy/inefficient in spewing API requests into the cloud). But for cost purposes with cloud storage, no microcharges to be found with Wasabi.

More details on our joint offering at:
https://wasabi.com/veeam-partner-solution/

[yasuda]

Hi Dan, do you have any comment on the previous discussion of immutable storage? Is Wasabi's immutable storage diferent from "...Amazon object-level immutability is more of a marketing term, in reality what they sell behind this term is regular object versioning..." ?

[dw432948jkk]

We are currently deciding on which back-up solution we will use. Although we like veeam better in on-premise backup, there is unfortunately no integrated cloud back-up solution yet. However, for a few months now, this connection has been offered by Altaro, which we also like, with the direct integration with Azure blobs as offsite backup.

There is always the request that veeam should offer this integrated. A few months ago, it was said that veeam is working on it and that it should be released in the fall.

Will there be this direct connection at short notice and if so which providers are supported (Azure, AWS an GCS would be great)?

Otherwise, we would probably soon decide for Altaro and would then be away as a potential customer.

[wishr]

Hi Peter,

Welcome to Veeam Community Forums and thanks for posting!

Please take a look at this and this posts as they should answer some of your questions. I've merged your post with the corresponding topic.

We already have a private beta of the next B&R version so you may jump in and give this functionality a try - please reach out to your local Veeam sales representatives to get access to it.

Regarding compatible providers, please check that thread.

Thanks

[dw432948jkk]

Ah, ok, that sounds good. I'll take a look at that.

Will existing customers with instance licenses also get a "free" upgrade to version 10?

[wishr]

Hi Peter,

Sure, as long as your subscription is still valid when v10 is released.

Thanks

[dw432948jkk]

How long will it take for version 10 to be released?