Using object storage as a backup target
wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr »

Hello,

Usually, we cannot comment on future release plans or product ETAs, but as Gostev mentioned in the July forum digest, it should be shipped before the end of this year if nothing unexpected pops up (particularly around 3rd party platform updates).

Thanks

dw432948jkk
Novice
Posts: 9
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » 1 person likes this post

yasuda wrote: May 29, 2019 5:15 pm Hi Dan, do you have any comment on the previous discussion of immutable storage? Is Wasabi's immutable storage diferent from "...Amazon object-level immutability is more of a marketing term, in reality what they sell behind this term is regular object versioning..." ?
Is there any reason why WasabiDan does not answer this question?

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

Because he did not log on here once after his only post above ;) I just checked his forum account.

In any case, I can answer this: Wasabi does not support S3 Object Lock at all at this time.

I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - just requires a lot more code from our side to work with, for obvious reasons. I mean - a single, actually immutable copy of an object that cannot be deleted or overwritten would have been so much simpler to architect against. While dealing with potentially deletable and overwriteable "immutable" objects requires tracking the required object versions in its version history (separately for each and every object), which adds significant complexity.

But anyway, it did not stop us :D

dw432948jkk
Novice
Posts: 9
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk »

Did I understand correctly that veeam is working to set up veeam-backups so that they can be copied (not moved) (with whatever other third party program) to the immutable storage of cloud backup providers like Azure, AWS, etc.? That would be great because, to my knowledge, no backup program can do this.

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!

yasuda
Enthusiast
Posts: 63
Liked: 10 times
Joined: May 15, 2014 3:29 pm
Full Name: Peter Yasuda
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by yasuda »

Gostev wrote: Sep 13, 2019 7:38 pm ...I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - ...
Thanks for the clarification!

Is it also true Glacier or Deep Archive data would be protected, because there is a minimum duration data has to be left there, before it can be deleted? Assuming you are only concerned with recovering your most recent backups.

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

My understanding is that Glacier has minimum duration charge, not minimum duration storage. Meaning, you can still delete data immediately after uploading, but you will be charged as if the data was stored there for a minimum required duration.

dw432948jkk
Novice
Posts: 9
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk »

Gostev wrote: Sep 15, 2019 10:32 pm Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!
The problem with this is the veeam backup chain metadata file.

Even with incremental backups, this file will always have the same name and will not be transferred by the programs that are supposed to do the uploads (rclone, cloudberry, etc.) because, for example, Azure reports back that the file already exists and can not be overwritten.

There is (unless you create a script that versioned this file before each upload and then always uploaded a modified version) from the perspective of upload programs no solution.

Is there a solution planned here from veeam?

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

You might be confusing Veeam with some other vendor, because Veeam names each incremental backup file uniquely.

dw432948jkk
Novice
Posts: 9
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk »

The files ending in .vib have different names, but one .vbm file always has the same name within a backup.

Well, I'm actually confused now :?

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » 1 person likes this post

VBM is a metadata file which is not essential for restore, as the actual data sits in VBK (full) and VIB (incremental) backup files.
Click Browse and select the necessary VBM or VBK file. If you select the VBM file, the import process will be notably faster. It is recommended that you select the VBK file only if the VBM file is not available.

dw432948jkk
Novice
Posts: 9
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk »

Ie. the vbm file is actually solely responsible for enabling a faster backup?

Otherwise, there are no disadvantages if you do not have the vbm file?

So far it has not emerged for me from the documents.

veremin
Product Manager
Posts: 17851
Liked: 1665 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by veremin » 1 person likes this post

It's not responsible for enabling faster backup, it makes backup import procedure work faster.

If you'd like to transfer backups offsite via 3-party utility, you can skip .vbm file. However, should disaster happen, you will need to import backup chain to backup server first (before you restore anything), without .vbm file this process will be significantly longer.

Hope it helps.

Thanks!

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

[MERGED] Veeam to Azure Blob

Post by BackupMonkey »

Hi all.

I'm just in the process of testing this and I'm completely confused.

Setup Azure Blob repository (AB1)
Setup local repository (LR1)
Setup Scale out repository to have LR1 and AB1 as members. Set it to move backups older than 0 days. (SOR1)
Setup job VM backup job (VMB1) to backup to SOR1
Setup a Backup Copy job (BCJ1) from VMB1 as source to SOR1 as target.

BCJ1 will fail with the error "Restore point is located in Backup Copy target repository and cannot be used as a source"
Assuming here that while Veeam understands tiers, backup copy jobs don't.

So,
Create another local repository (LR2)
Set VMB1 to use LR2
Set BCJ1 to use LR2 as source

BCJ1 will then copy the backups between the local repositories and yet again, nothing goes to azure. Grr


I can't backup direct to azure since I can't pick the AB1 as the primary destination for the backup jobs.
I can't do a backup copy job between tiers of the same scale out repository since the target and destinations have to be different and the job can't see the tiers.
I can't do a backup copy job between a local reposatory and a Azure repository because I can't select the azure repository.

No direct route & No backup copy route.

What am I missing here?

wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Veeam to Azure Blob

Post by wishr »

Hi,

You do not need copy jobs to offload your backups to Capacity Tier - it is done automatically using SOBR Offload Job, according to operational restore window.

This topic has been discussed many times - please review the documentation and posts above and let us know if you require additional information.

Thanks

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by BackupMonkey »

Hi wishr.

I've seen that thread and can I point out.
Gostev wrote: Oct 15, 2018 8:21 pm Sorry to hear that, Phil. As I've said, this is already planned for the future update based on the feedback from beta - but I do understand that you may not be willing to wait. I hope you will consider Veeam again once this feature is delivered!
It's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.

The offload doesn't happen automatically for me.
Are you saying that I have to completely disable the job before the SOBR will run? because being active but not transfering data doesn't appear to be enough, and if that is the case how do you SOBR backup jobs that have SQL transactional logging enabled?

wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr »

Capacity Tier was first introduced in B&R9.5 U4. The current version is 9.5U4b and the logic behind the purpose of this feature has not changed from that time.

To properly setup Capacity Tier I'd recommend referring to the Capacity Tier documentation, but in short words, the process looks like this:

1. You configure an Object Storage Repository;
2. Configure SOBR and add the aforementioned Object Storage Repository as Capacity Tier of SOBR;
3. Create a backup job targeting SOBR;
4. Offload job offloads created backups to Object Storage every 4 hours, according to the operational restore window you have configured. The same can be done manually when needed.

Thanks.

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » 1 person likes this post

BackupMonkey wrote: Oct 24, 2019 12:48 pmIt's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.
I was talking about v10. Yes, the year will indeed be enough :D however, so far we're only into the 10th month since our previous release, whereas our major release cadence is annual.

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by BackupMonkey »

@Wishr - I'm currently reading the Capacity Tier documentation again, I've clearly missed something. I can get the SOBR to run by hand just not automatically at this time. I'll track it down.
@Gostev - So next update then? :)

wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr »

I'd like to add that SQL t-log backup (.VLB) is not a part of a backup chain, thus will not be offloaded to Object Storage.

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

@BackupMonkey yes, this is a feature of v10, which we're planning to ship by the end of this year.

pstilber
Novice
Posts: 5
Liked: never
Joined: Jul 10, 2018 8:12 pm
Full Name: Phil Stilber
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by pstilber »

As an update to my response on page 1, I'd be fine with the capacity tier actually following the "move backups after 0 days" setting. As it is now that setting is very misleading as the active chain and the previous two backups aren't offloaded. I can't be the only person who incorrectly assumed "move after 0 days" did not mean "or after 9 days, whatevs".

I'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well. Basically, for DR purposes I want all of my backups in Azure ASAP, while keeping a week or two on site for faster restores over our most commonly requested RPOs. I don't particularly care if this is accomplished with a backup copy job or inside of the SOBR tiering itself; though I'd prefer inside of the SOBR tiering for fewer jobs to maintain.

I tried using an Azure File Share as a target for a backup copy job and the performance of that configuration was only slightly faster than copying the 1s and 0s by hand.

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

pstilber wrote: Oct 28, 2019 8:52 pmI'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well.
I'd prefer inside of the SOBR tiering for fewer jobs to maintain.
The good news is, this is exactly how v10 works.

vkky2k
Influencer
Posts: 12
Liked: never
Joined: Oct 23, 2019 6:10 pm
Full Name: rogerpei
Contact:

[MERGED] Use Cloud Object Storage for Veeam Repository

Post by vkky2k »

I understand I can configure AWS S3 or Azure Blob for one of Veeam repository. I probably should back up data to the local repository first, before tier the data to the cloud.

The question is, can I back up data directly to the cloud object storage, without staging in the local first?

Thanks!

wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Use Cloud Object Storage for Veeam Repository

Post by wishr »

Hi Roger,

I'm merging your topic with an existing thread - please take a look since it should answer all potential questions you may have in that regard.

Thanks

vkky2k
Influencer
Posts: 12
Liked: never
Joined: Oct 23, 2019 6:10 pm
Full Name: rogerpei
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by vkky2k »

I am not so clear on what the answer is. Can Veeam allow me to back up data to Cloud repository without going to a local repository?

Gostev
SVP, Product Management
Posts: 26723
Liked: 4281 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev »

No, you cannot backup directly to object storage.

fernbrun
Lurker
Posts: 2
Liked: never
Joined: Dec 18, 2019 7:52 am
Full Name: IT Departement
Contact:

[MERGED] Support to setup Scale-out Repository with Azure Backup

Post by fernbrun »

Hello,

I have currently setup Veeam backup with 2 local repositories (1 for backup and 1 for archive job) and also a remote archive repository (in case of disaster in our DC).

So far everything works, but I would like to setup a 3 point of backup in the cloud, in case of major disaster in the town where is my company.

I already created an azure object storage repository and I would like to simply always add the last backup point + 7 last incremental backups points.

I understood that for that I need to use SOR, but based on what I read the backup are uploaded to the cloud during the offload job for archiving, but what I would like to achieve is to keep the data in local repository as well.

Could you please help let me know if this is doable and if how to configure it ?

Thanks

wishr
Veeam Software
Posts: 1826
Liked: 208 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Support to setup Scale-out Repository with Azure Backup

Post by wishr »

Hi Fernbrun,

"Copy" to object storage functionality is coming in B&R v10 which is expected to be released soon. With 9.5 Update 4b, you can only "move" aging backups to Azure.

Thanks

Post Reply

Who is online

Users browsing this forum: dloseke and 12 guests