Discussions specific to object storage
wishr
Veeam Software
Posts: 1334
Liked: 133 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr » Sep 09, 2019 1:59 pm

Hello,

Usually, we cannot comment on future release plans or product ETAs, but as Gostev mentioned in the July forum digest, it should be shipped before the end of this year if nothing unexpected pops up (particularly around 3rd party platform updates).

Thanks

dw432948jkk
Novice
Posts: 8
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » Sep 13, 2019 12:49 pm 1 person likes this post

yasuda wrote:
May 29, 2019 5:15 pm
Hi Dan, do you have any comment on the previous discussion of immutable storage? Is Wasabi's immutable storage diferent from "...Amazon object-level immutability is more of a marketing term, in reality what they sell behind this term is regular object versioning..." ?
Is there any reason why WasabiDan does not answer this question?

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Sep 13, 2019 7:38 pm

Because he did not log on here once after his only post above ;) I just checked his forum account.

In any case, I can answer this: Wasabi does not support S3 Object Lock at all at this time.

I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - just requires a lot more code from our side to work with, for obvious reasons. I mean - a single, actually immutable copy of an object that cannot be deleted or overwritten would have been so much simpler to architect against. While dealing with potentially deletable and overwriteable "immutable" objects requires tracking the required object versions in its version history (separately for each and every object), which adds significant complexity.

But anyway, it did not stop us :D

dw432948jkk
Novice
Posts: 8
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » Sep 14, 2019 7:23 am

Did I understand correctly that veeam is working to set up veeam-backups so that they can be copied (not moved) (with whatever other third party program) to the immutable storage of cloud backup providers like Azure, AWS, etc.? That would be great because, to my knowledge, no backup program can do this.

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Sep 15, 2019 10:32 pm

Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!

yasuda
Enthusiast
Posts: 63
Liked: 10 times
Joined: May 15, 2014 3:29 pm
Full Name: Peter Yasuda
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by yasuda » Sep 16, 2019 3:10 pm

Gostev wrote:
Sep 13, 2019 7:38 pm
...I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - ...
Thanks for the clarification!

Is it also true Glacier or Deep Archive data would be protected, because there is a minimum duration data has to be left there, before it can be deleted? Assuming you are only concerned with recovering your most recent backups.

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Sep 17, 2019 9:00 am

My understanding is that Glacier has minimum duration charge, not minimum duration storage. Meaning, you can still delete data immediately after uploading, but you will be charged as if the data was stored there for a minimum required duration.

dw432948jkk
Novice
Posts: 8
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » Sep 24, 2019 8:07 pm

Gostev wrote:
Sep 15, 2019 10:32 pm
Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!
The problem with this is the veeam backup chain metadata file.

Even with incremental backups, this file will always have the same name and will not be transferred by the programs that are supposed to do the uploads (rclone, cloudberry, etc.) because, for example, Azure reports back that the file already exists and can not be overwritten.

There is (unless you create a script that versioned this file before each upload and then always uploaded a modified version) from the perspective of upload programs no solution.

Is there a solution planned here from veeam?

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Sep 24, 2019 9:56 pm

You might be confusing Veeam with some other vendor, because Veeam names each incremental backup file uniquely.

dw432948jkk
Novice
Posts: 8
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » Sep 25, 2019 5:03 am

The files ending in .vib have different names, but one .vbm file always has the same name within a backup.

Well, I'm actually confused now :?

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Sep 25, 2019 11:11 am 1 person likes this post

VBM is a metadata file which is not essential for restore, as the actual data sits in VBK (full) and VIB (incremental) backup files.
Click Browse and select the necessary VBM or VBK file. If you select the VBM file, the import process will be notably faster. It is recommended that you select the VBK file only if the VBM file is not available.

dw432948jkk
Novice
Posts: 8
Liked: 1 time
Joined: Sep 05, 2019 8:26 am
Full Name: Peter Müller
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by dw432948jkk » Sep 25, 2019 1:00 pm

Ie. the vbm file is actually solely responsible for enabling a faster backup?

Otherwise, there are no disadvantages if you do not have the vbm file?

So far it has not emerged for me from the documents.

veremin
Product Manager
Posts: 17073
Liked: 1475 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by veremin » Sep 25, 2019 4:22 pm 1 person likes this post

It's not responsible for enabling faster backup, it makes backup import procedure work faster.

If you'd like to transfer backups offsite via 3-party utility, you can skip .vbm file. However, should disaster happen, you will need to import backup chain to backup server first (before you restore anything), without .vbm file this process will be significantly longer.

Hope it helps.

Thanks!

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

[MERGED] Veeam to Azure Blob

Post by BackupMonkey » Oct 24, 2019 11:25 am

Hi all.

I'm just in the process of testing this and I'm completely confused.

Setup Azure Blob repository (AB1)
Setup local repository (LR1)
Setup Scale out repository to have LR1 and AB1 as members. Set it to move backups older than 0 days. (SOR1)
Setup job VM backup job (VMB1) to backup to SOR1
Setup a Backup Copy job (BCJ1) from VMB1 as source to SOR1 as target.

BCJ1 will fail with the error "Restore point is located in Backup Copy target repository and cannot be used as a source"
Assuming here that while Veeam understands tiers, backup copy jobs don't.

So,
Create another local repository (LR2)
Set VMB1 to use LR2
Set BCJ1 to use LR2 as source

BCJ1 will then copy the backups between the local repositories and yet again, nothing goes to azure. Grr


I can't backup direct to azure since I can't pick the AB1 as the primary destination for the backup jobs.
I can't do a backup copy job between tiers of the same scale out repository since the target and destinations have to be different and the job can't see the tiers.
I can't do a backup copy job between a local reposatory and a Azure repository because I can't select the azure repository.

No direct route & No backup copy route.

What am I missing here?

wishr
Veeam Software
Posts: 1334
Liked: 133 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Veeam to Azure Blob

Post by wishr » Oct 24, 2019 12:00 pm

Hi,

You do not need copy jobs to offload your backups to Capacity Tier - it is done automatically using SOBR Offload Job, according to operational restore window.

This topic has been discussed many times - please review the documentation and posts above and let us know if you require additional information.

Thanks

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by BackupMonkey » Oct 24, 2019 12:48 pm

Hi wishr.

I've seen that thread and can I point out.
Gostev wrote:
Oct 15, 2018 8:21 pm
Sorry to hear that, Phil. As I've said, this is already planned for the future update based on the feedback from beta - but I do understand that you may not be willing to wait. I hope you will consider Veeam again once this feature is delivered!
It's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.

The offload doesn't happen automatically for me.
Are you saying that I have to completely disable the job before the SOBR will run? because being active but not transfering data doesn't appear to be enough, and if that is the case how do you SOBR backup jobs that have SQL transactional logging enabled?

wishr
Veeam Software
Posts: 1334
Liked: 133 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr » Oct 24, 2019 1:07 pm

Capacity Tier was first introduced in B&R9.5 U4. The current version is 9.5U4b and the logic behind the purpose of this feature has not changed from that time.

To properly setup Capacity Tier I'd recommend referring to the Capacity Tier documentation, but in short words, the process looks like this:

1. You configure an Object Storage Repository;
2. Configure SOBR and add the aforementioned Object Storage Repository as Capacity Tier of SOBR;
3. Create a backup job targeting SOBR;
4. Offload job offloads created backups to Object Storage every 4 hours, according to the operational restore window you have configured. The same can be done manually when needed.

Thanks.

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Oct 24, 2019 1:14 pm 1 person likes this post

BackupMonkey wrote:
Oct 24, 2019 12:48 pm
It's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.
I was talking about v10. Yes, the year will indeed be enough :D however, so far we're only into the 10th month since our previous release, whereas our major release cadence is annual.

BackupMonkey
Influencer
Posts: 14
Liked: 4 times
Joined: May 23, 2018 12:14 pm
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by BackupMonkey » Oct 24, 2019 1:18 pm

@Wishr - I'm currently reading the Capacity Tier documentation again, I've clearly missed something. I can get the SOBR to run by hand just not automatically at this time. I'll track it down.
@Gostev - So next update then? :)

wishr
Veeam Software
Posts: 1334
Liked: 133 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by wishr » Oct 24, 2019 1:30 pm

I'd like to add that SQL t-log backup (.VLB) is not a part of a backup chain, thus will not be offloaded to Object Storage.

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Oct 24, 2019 2:25 pm

@BackupMonkey yes, this is a feature of v10, which we're planning to ship by the end of this year.

pstilber
Lurker
Posts: 2
Liked: never
Joined: Jul 10, 2018 8:12 pm
Full Name: Phil Stilber
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by pstilber » Oct 28, 2019 8:52 pm

As an update to my response on page 1, I'd be fine with the capacity tier actually following the "move backups after 0 days" setting. As it is now that setting is very misleading as the active chain and the previous two backups aren't offloaded. I can't be the only person who incorrectly assumed "move after 0 days" did not mean "or after 9 days, whatevs".

I'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well. Basically, for DR purposes I want all of my backups in Azure ASAP, while keeping a week or two on site for faster restores over our most commonly requested RPOs. I don't particularly care if this is accomplished with a backup copy job or inside of the SOBR tiering itself; though I'd prefer inside of the SOBR tiering for fewer jobs to maintain.

I tried using an Azure File Share as a target for a backup copy job and the performance of that configuration was only slightly faster than copying the 1s and 0s by hand.

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Oct 28, 2019 11:12 pm

pstilber wrote:
Oct 28, 2019 8:52 pm
I'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well.
I'd prefer inside of the SOBR tiering for fewer jobs to maintain.
The good news is, this is exactly how v10 works.

vkky2k
Influencer
Posts: 11
Liked: never
Joined: Oct 23, 2019 6:10 pm
Full Name: rogerpei
Contact:

[MERGED] Use Cloud Object Storage for Veeam Repository

Post by vkky2k » Nov 11, 2019 11:51 am

I understand I can configure AWS S3 or Azure Blob for one of Veeam repository. I probably should back up data to the local repository first, before tier the data to the cloud.

The question is, can I back up data directly to the cloud object storage, without staging in the local first?

Thanks!

wishr
Veeam Software
Posts: 1334
Liked: 133 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Use Cloud Object Storage for Veeam Repository

Post by wishr » Nov 11, 2019 12:08 pm

Hi Roger,

I'm merging your topic with an existing thread - please take a look since it should answer all potential questions you may have in that regard.

Thanks

vkky2k
Influencer
Posts: 11
Liked: never
Joined: Oct 23, 2019 6:10 pm
Full Name: rogerpei
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by vkky2k » Nov 18, 2019 8:06 pm

I am not so clear on what the answer is. Can Veeam allow me to back up data to Cloud repository without going to a local repository?

Gostev
SVP, Product Management
Posts: 25141
Liked: 3692 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 9.5 Update 4 and Microsoft Azure Blob Storage

Post by Gostev » Nov 19, 2019 2:05 am

No, you cannot backup directly to object storage.

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests