Feature Request: Azure Blob Storage

[nichola.vandevoorde]

Dear Sir,
Dear Madam,

I would like to file a feature request (couldn't find a previous request on the forum) regarding Veeam Cloud Connect backup to Azure Blob Storage (cool storage).

Now the amount of storage we can present to a Veeam Cloud Connect infrastructure is dependent of the Azure VM size (max 64 disks -> high cost for this VM). It would be nice to offload backup data from on-prem to Azure via the Veeam Cloud Connect infrastructure OR directly to Azure Storage blob account (like in Veeam Cloud Edition).

Thanks in advance!

Nichola

[Mike Resseler]

Hi Nichola,

One of the biggest issues I see with that request is that it can take ages to download the backup again and recover... When there is no compute attached to the storage, you can't do item level recovery, and downloading a full backup to restore one file will be costly + if there is a real disaster, downloading all backups will significantly slow down the recovery process. It could take days or longer before you can even start recovering something.

So if you talk about placing backups onto Blob storage, can you live with the above? Or are you seeing it differently (then just putting the backups on that storage)

Let me know

Mike

[veremin]

Now the amount of storage we can present to a Veeam Cloud Connect infrastructure is dependent of the Azure VM size (max 64 disks -> high cost for this VM).

It's dependent only if you stick to appliance that is shipped in form of Azure VM (available in Azure Marketplace).

It would be nice to offload backup data from on-prem to Azure via the Veeam Cloud Connect infrastructure OR directly to Azure Storage blob account (like in Veeam Cloud Edition).

Are you a SP yourself? Will you be keen to sacrifice restore time, miss SLA, etc. at the cost of increased disk space?

Thanks.

[csinetops]

I'd agree with Mike, we use Azure blob storage but it is only for long term retention or if our main site completely burned to the ground. In that case, a week to recover while we download backups wouldn't really matter because we'd have to rebuild from scratch.

We front end our Azure with a Netapp AltaVault appliance. That takes care of dedupe/compression and moving the data into Azure. There is also a AltaVault cloud appliance that you can spin up, inventory your backups and restore right to Azure if you want.

We also use Cloud Connect with a Veeam partner for one of our branches, that works well if you don't have a ton of data and just need some backups off site.

I guess what are your goals?

[Gostev]

Will the upcoming 32TB virtual disks for Azure VMs solve your concern?

[fredpelaez]

Hi,

I would agree with Nichola that this feature is a must. Backup data in Azure Blob Storage would be for long term retention (i.e tape replacement). I do agree that MS StorSimple and an NTAP Altavault can push data directly to Blob Storage quite efficiently but there is a cost element to it. Azure VM's don't really do the job because of the capacity limit. I would prefer to have a storage with "unlimited bucket" that I can use. CommVault Simpana can do this natively. I did hear that Microsoft DPM can do this as well (can't verify this at this point though)

[veremin]

Azure VM's don't really do the job because of the capacity limit.

But what do you think about forthcoming disk capacity increase mentioned by Anton? Thanks.

[fredpelaez]

In my opinion, 32 TB is too small.

[veremin]

It's 32 TB per virtual disk. In other words, 32x increase of currently existing disk limit (1024 GB per one virtual disk) won't answer your requirements? May I ask you how much data do you plan to store in public cloud? Thanks.

[nichola.vandevoorde]

Will the upcoming 32TB virtual disks for Azure VMs solve your concern?

That will cover our issue with the capacity limit in combination with Veeam CC on Azure.

Hi Nichola,

One of the biggest issues I see with that request is that it can take ages to download the backup again and recover... When there is no compute attached to the storage, you can't do item level recovery, and downloading a full backup to restore one file will be costly + if there is a real disaster, downloading all backups will significantly slow down the recovery process. It could take days or longer before you can even start recovering something.

So if you talk about placing backups onto Blob storage, can you live with the above? Or are you seeing it differently (then just putting the backups on that storage)

Let me know

Mike

You are absolutely right with the missing restore capabilities in case it's a simple offload to a blob storage account. However, the majority of our customers do not have large amounts of data and want to get rid of their tape handling in a "the most economical way" (which I do not always agree with). So the most economical way in their eyes is to have a simple long term retention backup copy job to cool Azure blob storage (being aware of the limited RTO and restore capabilities).

Hi,

I would agree with Nichola that this feature is a must. Backup data in Azure Blob Storage would be for long term retention (i.e tape replacement). I do agree that MS StorSimple and an NTAP Altavault can push data directly to Blob Storage quite efficiently but there is a cost element to it. Azure VM's don't really do the job because of the capacity limit. I would prefer to have a storage with "unlimited bucket" that I can use. CommVault Simpana can do this natively. I did hear that Microsoft DPM can do this as well (can't verify this at this point though)

I can verify that SCDPM has this capability (so does Azure Backup Server). I agree with the cost element, which can be a showstopper in some cases.

Sorry for the late response and thanks for the rapid feedback!

[Mike Resseler]

Thanks all for the information

Really appreciated, I won't make any promises of course but these insights give us valuable information at various discussion tables

Cheers
Mike

[mtunc]

May I ask - approximately when will the 32TB per disk feature be released? Are we talking in a few weeks or few months? I tried to find some documentation on this but couldn't find any.

This week I have been playing around with Azure and the 1TB per disk limitation (and 32 disks per VM which costs upwards of £800 per month!) is putting me off.

Also may I ask about the "Veeam Cloud Connect for the Enterprise" route in Azure? Is that essentially running your own Service Provider instance in Azure without paying a 3rd party to run it all for you?

Thanks!
Mikail

[Mike Resseler]

Mikail,

Unfortunately I can't give you a timeline for the 32TB disk feature. Microsoft has promised that they are working on it (as we speak) but when they go live is not known. There is no documentation on it but Mark Russinovich has stated it in public at Ignite during a session.

We aware of the costly VM that comes with 32 disks and this is one of the discussion points which we have with Microsoft Azure teams on regular basis. So they are also aware of it and understand it very well.

For the Veeam Cloud Connect for the Enterprise route. In essence this is indeed "playing" your own (internal) Service Provider for your enterprise. We see this as a solution that can solve many issues in large enterprises but the biggest problem that it potentially could solve is RoBo offices that have their own local backup for the few servers they run and then get a copy offsite to a central environment, managed by the global IT team. In that aspect, we are also building the Veeam Availability Console that can be used as an addition to VCC-E but also give "managed" service provider capabilities to remotely manage and monitor those local Veeam Backup instances without the need of complex networking.

PS: This does not apply to Azure alone... VAC will also be able to be installed on-premises if you don't want to go the Azure route but want to keep it in your central datacenter

Hope it helps
Mike

[bJako]

You can create file share in Azure and use it on Veeam server - create new repository, type= Shared Folder, \\yourstoragename.file.core.windows.net\yoursharename and you put credentials to it.
Size is max 5 TB and fileshare type storage is most expensive in Azure.
Full 5TB will cost ~350 €/month.

But you can use it as normal repository, file and app level resore works, even Instant Recovery seems to work (very slow of course).

Backup/restore speed depends on your internet connection and connection between your ISP and Azure DC.

First full Copy Job backup runs ~50Mbit/s (5 MB/s). Bottleneck - Target (99%) (my backup testsite have Internet connection 100 Mbit/s)

What Veeam experts think about this method?

[Mike Resseler]

Jako,

We have investigated this and we see a few issues.

First, if MSFT can improve this technology (or method ) then it certainly becomes a good and valid one. But...

* Yes, 5TB is the max fileshare and the biggest file you can use is 1 TB (so even with per-VM backup chains, this could get many customers into trouble...)
* The price is indeed pretty high.

The way I see it, this can become better when MSFT increases the max size per share and the max size per file (these numbers are not random... So every time MSFT makes a change somewhere (for example the 32TB disks soon...) I expect these numbers to change also. What's more, I start to notice a trend that higher capacity and better performance and so on does not always result in a more expensive price. So we probably have some nice things coming up (This is not a FACT by the way, I am guessing here and looking at the patterns of the last few years...)

Now, final thoughts is indeed performance. As you said yourself, the biggest bottleneck is the target but if this is for your backup copy job or secondary backup, and you can live with the lower performance, I see no reasons to say NO against it.

Just my thoughts

Mike

[mtunc]

For the Veeam Cloud Connect for the Enterprise route. In essence this is indeed "playing" your own (internal) Service Provider for your enterprise. We see this as a solution that can solve many issues in large enterprises but the biggest problem that it potentially could solve is RoBo offices that have their own local backup for the few servers they run and then get a copy offsite to a central environment, managed by the global IT team. In that aspect, we are also building the Veeam Availability Console that can be used as an addition to VCC-E but also give "managed" service provider capabilities to remotely manage and monitor those local Veeam Backup instances without the need of complex networking.

Hello Mike,
Thanks for the prompt reply.

With regards to the Cloud Connect for Enterprise - why is this limited to only Enterprise VMWare/Microsoft customers? I'm just wondering as this sounds like a good solution except we can't get it because we do not have a VMware enterprise license OR a Microsoft enterprise agreement.

[njsmee]

I think that the main point is being missed here, the original question was can we use Blob Storage and if not when can we, as attached disks to a large VM are expensive and that is indeed a showstopper in many cases, there was a previous comment made regarding the use of Azure for long term retention and that is certainly what we are looking at using it for, so the speed of recovery is not the highest priority.

[veremin]

the original question was can we use Blob Storage

Currently, you cannot.

if not when can we

At this moment we cannot provide any feedback regarding next release feature set.

[rriiicchh]

I'd just like to add my voice to this.

The main use case for us is speed for offsiting. We have a tape server which we run tape jobs at the weekend but that means the backup files are sitting on the onsite server for up to 7 days. Whereas if we could push backups to Azure then they could be almost immediately offsite.

In the event of a total site loss we could download the blobs into a VM pretty quickly (typically we see 3GB/s connections from blob storage to our Azure VMs G3-5) so we could be back up and running very quickly even if we had to download from blob, import into a Veeam Server in Azure and then restore. Would be quicker than finding/buying an unmelted tape drive and then copying and importing on a server and restoring.

Pushing these to blob storage gives us a quick route to off-siting the backups onto a different network (anything destroys local environment from fire through to malware) so not impact this, plus reduces costs by not having to have a VM run all the time and/or Cloud Connect licensing which means this probably won't be a goer as it is undercutting a Veeam license.

We achieve this now by azcopy and scheduled tasks (not really a great solution).

[Mike Resseler]

We just announced an archive tier that will support Azure blob storage coming in v10. Literally minutes ago on stage at VeeamON

[rriiicchh]

Now that is a quick turn around

If you respond that quickly shall I start listing all my other features requests

[Mike Resseler]

Lol. You can try, maybe we actually announced it today, or announce it tomorrow