Discussions specific to object storage
MikePettit
Influencer
Posts: 11
Liked: never
Joined: Jun 04, 2015 11:56 pm
Full Name: Michael Pettit
Contact:

9.5 Update 4 - Azure Blob Storage

Post by MikePettit » Oct 08, 2018 5:19 pm

After some reading, I'm getting the feeling that Update 4, while it will allow you to connect to Azure blob storage (I would like to connect to the Cool option at $10 per TB), it will only allow you to connect to it as an "Archive" repository. And archive repositories won't allow us to select them as a primary backup target or backup copy job target, but instead be the back-end of a scale-out repository. And the only problem with that is that in order for your backup data to start copying to the Archive tier in your scale-out repository, you would have to run a full backup so that everything before that full backup would start copying. However, the latest full backup in that scenario isn't also copied, it just sits on your primary target storage with all incrementals until another full backup is taken and "closes the old chain".

If I understand this right, in order to keep copying to the archive tier each day and staying current, I'd need to run a full backup each day, which would be a ridiculous backup strategy. So I'm hoping I'm either understanding this incorrectly and you can actually point to the archive tier in a backup job or backup copy job, or you can add Azure blob as a "Service Provider" so that you can point to it directly from a backup job.

Does anyone from Veeam have any information on this?

Thank you!

Rick.Vanover
Veeam Software
Posts: 553
Liked: 117 times
Joined: Nov 30, 2010 3:19 pm
Full Name: Rick Vanover
Location: Columbus, Ohio USA
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Rick.Vanover » Oct 08, 2018 5:41 pm

Hey Mike - the logic here is that the object storage implementation is going to be a capacity tier for the Scale-Out Backup Repository.

It is indeed part of the Scale-Out Backup Repository, and you can think of it as a way to make the SOBR infinite. The magical part here, aside from the how the files are stored bit, is that it is completely configurable. Specifically, there will be an option to specify "how many days" before backups are sent to the capacity tier. This specification can also be set to zero - meaning - once a backup is on disk, it is sent to the object storage bucket/folder.

What this implementation will not do - is backup only to object storage.

The story here that I described is part of 9.5 Update 4. You can reach out to your Veeam SE for the beta or more information.

MikePettit
Influencer
Posts: 11
Liked: never
Joined: Jun 04, 2015 11:56 pm
Full Name: Michael Pettit
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by MikePettit » Oct 08, 2018 7:55 pm

Thank you, Rick! I appreciate your response very much. I do have the beta and have tested the 0 day setting for the capacity tier, but unfortunately it only appears to copy data to the capacity tier for any data prior to the last full backup (closing the last chain of Full/Incrementals). In my opinion, that is very unfortunate. Perhaps it'll change when the update is live.

Thank you

Rick.Vanover
Veeam Software
Posts: 553
Liked: 117 times
Joined: Nov 30, 2010 3:19 pm
Full Name: Rick Vanover
Location: Columbus, Ohio USA
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Rick.Vanover » Oct 09, 2018 1:36 am

Can you make synthetic (or active) fulls happen more frequently?

Gostev
SVP, Product Management
Posts: 23624
Liked: 3119 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Gostev » Oct 09, 2018 9:20 am 1 person likes this post

Mike, yes you're basically misunderstanding U4 functionality as a whole, trying to use it for something it is not designed to do. I am sorry if this was not correctly explained to you by your Veeam sales engineer who provided you the beta code.

U4 functionality is about adding Capacity Tier to your scale-out backup repositories to allow offloading older backup files into the cheaper storage. This reduces your long-term retention costs and makes your scale-out backup repository capacity virtually infinite, simplifying storage management - but it does not change anything else in the bigger picture. Particularly, your still need to create copies of your backups to another backup repository normally, just as before.

You seem to be trying to repurpose this functionality to COPY the latest (not MOVE the oldest) backups to the object storage immediately as they're created. This is simply not what this functionality is designed for. Also, we recommend that your most recent backups (last 7-14 days) remain on-prem to provide for fast restores. Offloading latest backups right away is generally not so good idea from cost perspective - while cold object storage itself can indeed be quite cheap indeed, you will find that restores from it are quite expensive (this is exactly where cloud vendors "get you", so to speak).

MikePettit
Influencer
Posts: 11
Liked: never
Joined: Jun 04, 2015 11:56 pm
Full Name: Michael Pettit
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by MikePettit » Oct 10, 2018 7:31 pm

Thank you for the excellent information! Yes, you're understanding exactly what I'm trying to do. I can see now that I shouldn't be using the archive tier for this purpose.

Right now I have COPY backup jobs sending the most recent backup data to an Azure VM with Azure disks attached to it. The ultimate goal is to get these COPY jobs to send the data to Azure BLOB, by any means necessary, because it's so much cheaper. So I've been trying to get something to work so that the Azure BLOB repository can be a target for my COPY job, but can't find a way. Do you know if Azure BLOB can be setup as a "Service Provider"? I know that Service Provider storage can be setup as a primary target for jobs.

I want to do backups onsite, but have them replicate to the cloud right after the job is done. Right now that's being performed by COPY jobs, but I would love to send those COPY jobs straight to Azure BLOB or Wasabi's S3 Storage. But because these don't seem to be configurable for anything but the archive tier, I'll never be able to do that. Am I wrong? Is there a way to do what I want to do?

Backup Job to Local Storage > Copy Job to Azure BLOB Storage (Or Wasabi S3-Generic Storage)

Thank you!

Rick.Vanover
Veeam Software
Posts: 553
Liked: 117 times
Joined: Nov 30, 2010 3:19 pm
Full Name: Rick Vanover
Location: Columbus, Ohio USA
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Rick.Vanover » Oct 10, 2018 10:06 pm

The implementation will always have first step of the backups be on a non-Azure (or S3) repository - with the whole logic being what is stored in object storage to be the backups that are outside of normal operational restore windows.

And Mike - that's the logic - the restore with Veeam needs to continue to be an awesome experience. If the most recent backup is in the object storage - that would not necessarily be the case.

MikePettit
Influencer
Posts: 11
Liked: never
Joined: Jun 04, 2015 11:56 pm
Full Name: Michael Pettit
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by MikePettit » Oct 10, 2018 11:43 pm

Thank you, Rick! Any way for me to get a COPY job to point directly to Azure Blob storage? Right now I point my COPY job to an Azure VM with lots of disks attached, a much more expensive method.

Thank you!

Rick.Vanover
Veeam Software
Posts: 553
Liked: 117 times
Joined: Nov 30, 2010 3:19 pm
Full Name: Rick Vanover
Location: Columbus, Ohio USA
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Rick.Vanover » Oct 11, 2018 1:43 am

The file copy job and backup copy jobs as implemented in the current beta do not allow that. The whole premise of our implementation is to leave a shell on the regular repository. The shell is just a shadow of the full data set. The bulk of the data is placed in the object storage resource. The shell on-premises has the metadata of the job and the locales of the full data set. The way this is implemented is efficient for many reasons.

I really think what you want is "WS FTP for the Cloud". A partner of sorts of Veeam has that capability https://www.cloudberrylab.com/explorer/ ... azure.aspx.

Gostev
SVP, Product Management
Posts: 23624
Liked: 3119 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Gostev » Oct 14, 2018 10:33 pm 1 person likes this post

MikePettit wrote:
Oct 10, 2018 11:43 pm
Any way for me to get a COPY job to point directly to Azure Blob storage? Right now I point my COPY job to an Azure VM with lots of disks attached, a much more expensive method.
It's planned for the future updates. You will still need a regular repository for the "Performance Tier" of your scale-out backup repository for all the reasons mentioned above, however the "Capacity Tier" will have an option to duplicate any created backups into object storage as soon as they are created (as opposed to moving them there as they age out of your operational restores window). So this is a different use case from the hierarchical storage management we're bringing in Update 4, but we do understand this other use case you're talking about as well.

gummett
Veeam Software
Posts: 27
Liked: 5 times
Joined: Jan 30, 2017 9:23 am
Full Name: Ed Gummett
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by gummett » Oct 15, 2018 8:18 am

Hi Mike,
The ultimate goal is to get these COPY jobs to send the data to Azure BLOB, by any means necessary
Just checking you're aware that you could achieve your goal today using a VTL? Full details available at https://www.veeam.com/blog/archiving-to ... d-vtl.html
Ed Gummett
Systems Engineer UK & Ireland
Veeam Software

m.novelli
Veeam ProPartner
Posts: 346
Liked: 40 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by m.novelli » Oct 15, 2018 8:33 am 1 person likes this post

Wow, this is a sad news, I expected Veeam Backup Copy Job to work natively with Azure Blob Storage

Marco

dimaslan
Service Provider
Posts: 40
Liked: 7 times
Joined: Jul 01, 2017 8:02 pm
Full Name: Dimitris Aslanidis
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by dimaslan » Oct 15, 2018 2:27 pm

Would it work if one would deploy an Azure VM with say 60 GB of "normal" storage for the OS and Veeam installation and than attach a 10 or 20 TB disk of Cold Blob storage? If we would set this up as a Cloud Connect server, can we setup backup copy jobs go there? I am mostly interested if this blob storage can be "seen" as a normal backup repository so that backup copy jobs going to that CC Azure server can use it directly.

m.novelli
Veeam ProPartner
Posts: 346
Liked: 40 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by m.novelli » Oct 15, 2018 2:39 pm

Don't like this workaround because VM in Azure are quite pricey and have to be maintained (a little) over time

I would love to use directly Azure low cost storage from Veeam B&R

My SMB italian customers can afford an expense of 2 - 300 euro / month to export their backups to Azure, more will fail the project (my typical full backup is around 1,5 - 2,0 TB)

Even more important: the Backup Copy Job to Azure feature should be included in Veeam B&R Standard license. Having to buy the Enterprise licenze will fail again the project in my SMB user base

Marco

Gostev
SVP, Product Management
Posts: 23624
Liked: 3119 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 - Azure Blob Storage

Post by Gostev » Oct 15, 2018 2:50 pm 1 person likes this post

Hi Marco, just for your planning purposes, due to being a feature of Scale-out Backup Repository that is only available in the Enterprise Edition, it is safe to bet that object storage integrations will not be available in the Standard Edition.

I really hope "Standard" will be renamed to "Basic" one day, because it's what it really is - most basic, no-thrills backup to a regular storage. We can only keep its price so low (3x cheaper than Enterprise Plus) by ensuring there's little functionality we need to provide technical support for.

Thanks!

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests