9.5 Update 4 and Microsoft Azure Blob Storage

[MikePettit]

After some reading, I'm getting the feeling that Update 4, while it will allow you to connect to Azure blob storage (I would like to connect to the Cool option at $10 per TB), it will only allow you to connect to it as an "Archive" repository. And archive repositories won't allow us to select them as a primary backup target or backup copy job target, but instead be the back-end of a scale-out repository. And the only problem with that is that in order for your backup data to start copying to the Archive tier in your scale-out repository, you would have to run a full backup so that everything before that full backup would start copying. However, the latest full backup in that scenario isn't also copied, it just sits on your primary target storage with all incrementals until another full backup is taken and "closes the old chain".

If I understand this right, in order to keep copying to the archive tier each day and staying current, I'd need to run a full backup each day, which would be a ridiculous backup strategy. So I'm hoping I'm either understanding this incorrectly and you can actually point to the archive tier in a backup job or backup copy job, or you can add Azure blob as a "Service Provider" so that you can point to it directly from a backup job.

Does anyone from Veeam have any information on this?

Thank you!

[Rick.Vanover]

Hey Mike - the logic here is that the object storage implementation is going to be a capacity tier for the Scale-Out Backup Repository.

It is indeed part of the Scale-Out Backup Repository, and you can think of it as a way to make the SOBR infinite. The magical part here, aside from the how the files are stored bit, is that it is completely configurable. Specifically, there will be an option to specify "how many days" before backups are sent to the capacity tier. This specification can also be set to zero - meaning - once a backup is on disk, it is sent to the object storage bucket/folder.

What this implementation will not do - is backup only to object storage.

The story here that I described is part of 9.5 Update 4. You can reach out to your Veeam SE for the beta or more information.

[MikePettit]

Thank you, Rick! I appreciate your response very much. I do have the beta and have tested the 0 day setting for the capacity tier, but unfortunately it only appears to copy data to the capacity tier for any data prior to the last full backup (closing the last chain of Full/Incrementals). In my opinion, that is very unfortunate. Perhaps it'll change when the update is live.

Thank you

[Rick.Vanover]

Can you make synthetic (or active) fulls happen more frequently?

[Gostev]

Mike, yes you're basically misunderstanding U4 functionality as a whole, trying to use it for something it is not designed to do. I am sorry if this was not correctly explained to you by your Veeam sales engineer who provided you the beta code.

U4 functionality is about adding Capacity Tier to your scale-out backup repositories to allow offloading older backup files into the cheaper storage. This reduces your long-term retention costs and makes your scale-out backup repository capacity virtually infinite, simplifying storage management - but it does not change anything else in the bigger picture. Particularly, your still need to create copies of your backups to another backup repository normally, just as before.

You seem to be trying to repurpose this functionality to COPY the latest (not MOVE the oldest) backups to the object storage immediately as they're created. This is simply not what this functionality is designed for. Also, we recommend that your most recent backups (last 7-14 days) remain on-prem to provide for fast restores. Offloading latest backups right away is generally not so good idea from cost perspective - while cold object storage itself can indeed be quite cheap indeed, you will find that restores from it are quite expensive (this is exactly where cloud vendors "get you", so to speak).

[MikePettit]

Thank you for the excellent information! Yes, you're understanding exactly what I'm trying to do. I can see now that I shouldn't be using the archive tier for this purpose.

Right now I have COPY backup jobs sending the most recent backup data to an Azure VM with Azure disks attached to it. The ultimate goal is to get these COPY jobs to send the data to Azure BLOB, by any means necessary, because it's so much cheaper. So I've been trying to get something to work so that the Azure BLOB repository can be a target for my COPY job, but can't find a way. Do you know if Azure BLOB can be setup as a "Service Provider"? I know that Service Provider storage can be setup as a primary target for jobs.

I want to do backups onsite, but have them replicate to the cloud right after the job is done. Right now that's being performed by COPY jobs, but I would love to send those COPY jobs straight to Azure BLOB or Wasabi's S3 Storage. But because these don't seem to be configurable for anything but the archive tier, I'll never be able to do that. Am I wrong? Is there a way to do what I want to do?

Backup Job to Local Storage > Copy Job to Azure BLOB Storage (Or Wasabi S3-Generic Storage)

Thank you!

[Rick.Vanover]

The implementation will always have first step of the backups be on a non-Azure (or S3) repository - with the whole logic being what is stored in object storage to be the backups that are outside of normal operational restore windows.

And Mike - that's the logic - the restore with Veeam needs to continue to be an awesome experience. If the most recent backup is in the object storage - that would not necessarily be the case.

[MikePettit]

Thank you, Rick! Any way for me to get a COPY job to point directly to Azure Blob storage? Right now I point my COPY job to an Azure VM with lots of disks attached, a much more expensive method.

Thank you!

[Rick.Vanover]

The file copy job and backup copy jobs as implemented in the current beta do not allow that. The whole premise of our implementation is to leave a shell on the regular repository. The shell is just a shadow of the full data set. The bulk of the data is placed in the object storage resource. The shell on-premises has the metadata of the job and the locales of the full data set. The way this is implemented is efficient for many reasons.

I really think what you want is "WS FTP for the Cloud". A partner of sorts of Veeam has that capability https://www.cloudberrylab.com/explorer/ ... azure.aspx.

[Gostev]

Any way for me to get a COPY job to point directly to Azure Blob storage? Right now I point my COPY job to an Azure VM with lots of disks attached, a much more expensive method.

It's planned for the future updates. You will still need a regular repository for the "Performance Tier" of your scale-out backup repository for all the reasons mentioned above, however the "Capacity Tier" will have an option to duplicate any created backups into object storage as soon as they are created (as opposed to moving them there as they age out of your operational restores window). So this is a different use case from the hierarchical storage management we're bringing in Update 4, but we do understand this other use case you're talking about as well.

[gummett]

Hi Mike,

The ultimate goal is to get these COPY jobs to send the data to Azure BLOB, by any means necessary

Just checking you're aware that you could achieve your goal today using a VTL? Full details available at https://www.veeam.com/blog/archiving-to ... d-vtl.html

[m.novelli]

Wow, this is a sad news, I expected Veeam Backup Copy Job to work natively with Azure Blob Storage

Marco

[dimaslan]

Would it work if one would deploy an Azure VM with say 60 GB of "normal" storage for the OS and Veeam installation and than attach a 10 or 20 TB disk of Cold Blob storage? If we would set this up as a Cloud Connect server, can we setup backup copy jobs go there? I am mostly interested if this blob storage can be "seen" as a normal backup repository so that backup copy jobs going to that CC Azure server can use it directly.

[m.novelli]

Don't like this workaround because VM in Azure are quite pricey and have to be maintained (a little) over time

I would love to use directly Azure low cost storage from Veeam B&R

My SMB italian customers can afford an expense of 2 - 300 euro / month to export their backups to Azure, more will fail the project (my typical full backup is around 1,5 - 2,0 TB)

Even more important: the Backup Copy Job to Azure feature should be included in Veeam B&R Standard license. Having to buy the Enterprise licenze will fail again the project in my SMB user base

Marco

[Gostev]

Hi Marco, just for your planning purposes, due to being a feature of Scale-out Backup Repository that is only available in the Enterprise Edition, it is safe to bet that object storage integrations will not be available in the Standard Edition.

I really hope "Standard" will be renamed to "Basic" one day, because it's what it really is - most basic, no-thrills backup to a regular storage. We can only keep its price so low (3x cheaper than Enterprise Plus) by ensuring there's little functionality we need to provide technical support for.

Thanks!

[m.novelli]

I'm super very sad Gostev

Marco

[pstilber]

Adding my frustration that there is no option for using Azure blob storage as a backup copy target. We were really excited for Azure blob support in Veeam, but after waiting nearly a year for it the implementation in this release has made us start shopping for a new backup vendor.

[Gostev]

Sorry to hear that, Phil. As I've said, this is already planned for the future update based on the feedback from beta - but I do understand that you may not be willing to wait. I hope you will consider Veeam again once this feature is delivered!

[MikePettit]

Gostov and others, while I'm really disappointed that I'm not going to easily be able to use Veeam to connect to Azure Blob with copy jobs, I really appreciate you taking the time to respond and let me know about VTL options and the fact that future updates will include that auto-copy-to-archive-after-initial-backup option. I haven't liked the idea of using VTL options for methods that I don't really consider tape-type operations, so I won't go down that road.

I will just eagerly be awaiting the next update it seems. Unless, of course, we can sneakily slide this very important option into update 4...

Thanks, guys! Much appreciated.

[m.novelli]

This topic is becoming hot, seems many users (as me) expected Veeam to support natively Backup Copy Jobs to Azure when they announced "Azure low cost storage support"

Marco

[Gostev]

Thanks all for your interest and validating the need. We will try to release the next update as fast as we can and this particular functionality is already implemented in the next update branch, so it's pretty much guaranteed to make it.

[JHuston]

Add me on as a user that needs native backup copy to Azure sooner rather than later. Since it was announced that the NetApp AltaVault is end of sale in Dec, we're looking to move off that platform asap. Yes there are other options such as VTL but its just something else to maintain, another layer to cause issues and it's clunky in my opinion. Cloud connect isn't a option for companies that need long term retention for lots of data. ( think 7+ years and multi petabytes of data).

I LOVE, LOVE, LOVE Veeam and would never want to stray to something else. But I have been pressured into looking at other platforms to fill our gaps and am impressed on what other vendors can do native in the cloud. Globally deduped and compressed backup sets right to Azure. Single file restores from Azure BLOB data right in the cloud, starting VM's right in the cloud fro mbackup sets. IMO These are all features Veeam needs to stay top dog, especially in the SMB space where we need our backup solution to be agile and be able to support rapid business changes and growth. With the cloud quickly becoming some part of SMB BUR/DR workflow, this would be of huge value to us.

[networkup]

Hi,

We need to make a plan on our hardware as the tape drive which we use for offsite backups is beginning to get errors.

Our Manager wants us to move away from tapes to cloud backups.

We need to be able to go direct to Azure blob storage from on premise Veeam backup and replication, when will this be possible? Any version number and estimated dates?

We need to make a business decision on whether to stay with Veeam and this is difficult without timescales on this feature.

Any help would be appreciated please.

Thanks

[mcz]

It's planned for the future updates. You will still need a regular repository for the "Performance Tier" of your scale-out backup repository for all the reasons mentioned above, however the "Capacity Tier" will have an option to duplicate any created backups into object storage as soon as they are created

Anton, what about insider-protection-features? You know a copied backup in Azure (or any other cloud) is great but can still be droped/modified/manipulated/destroyed by humans/malware which would always end in desaster. Some kind of insider-protection would be necessary to turn this feature into something like air-gap...

What do you think?
Thanks!

[Gostev]

Michael, what about setting a time-based retention lock on the storage container?
https://docs.microsoft.com/en-us/azure/ ... le-storage

[MikePettit]

Thanks, Gostov.

I've read about WORM and, while that has some benefits, I would much rather just keep my same strategy of backups. Right now I run copy backups to an Azure VM (with data disks attached) and tell it to keep 4 restore points. And in order for it to stick at 4 restore points, it has to do a merge process on the data once the 5th data set is copied, so I need the data to be writable. And I'm not even sure if the merge process is going to be a possibility in the blob world, but I can't really test until I can get there with a copy job. I'm in the same boat as others, where management is asking about other backup vendors who are better prepared for cloud. I hope to stick with Veeam, and if I'm able to then my first two things with update X will be:

1. Get a copy job running to azure blob.
2. See if the merge process works so I can keep only a few restore points up there.

[Gostev]

There are no merges with our object storage integration... we operate with pointers to data blocks - same concept as our current ReFS integration feature, if you're familiar with one.

[csydas]

For those of you looking to put backup (copies?) up into Azure, can I ask what your budget is for the egress costs? Or how you envision it going in a disaster recovery event?

I've worked with a client that was obsessed with keeping backups in Azure via a StoreSimple device, and we ended up with corrupted/unreadable backups from the cloud as a result with exceptionally long recovery time. I get that Amazon/Microsoft aren't going anywhere soon, but I just get the feeling that it's such a burden to recover from the majority of emergency restore scenarios, and I'm not sure I see the benefit unless you're willing to commit a significant budget to this. I have colleagues from abroad who are 100% in on Azure, and I just cannot fathom keeping any backups I rely on there.

I've rambled a bit, but what is the disaster recovery plan/budget you've all set for Azure/AWS restores?

[dimaslan]

CSydas,

Since the Azure cold blob storage is considered by most as a long term, archival option (monthlies, yearlies), then the prospect of it being a part of a DR scenario does not really apply.

I think that the likelihood of needing a restore from 2 or more years ago would be a one or two time occurrence for most.

[mcz]

Some of you already pointed out that restore/IR from azure blob storage would be quite paintful in terms of availability/performance - of course bandwith and latency cannot be as good as if you would use your backup on premises.

I'm woundering if it

a) technically would already be possible to do a restore/IR of a VM directly to azure with "full speed", bypassing slower internet links (so that the data would be directly written/read from azure to azure) and
b) if there are plans on veeam-side to provide such a feature in the near future

Thanks