9.5 Update 4 and Microsoft Azure Blob Storage

[wishr]

Hello,

Usually, we cannot comment on future release plans or product ETAs, but as Gostev mentioned in the July forum digest, it should be shipped before the end of this year if nothing unexpected pops up (particularly around 3rd party platform updates).

Thanks

[dw432948jkk]

Hi Dan, do you have any comment on the previous discussion of immutable storage? Is Wasabi's immutable storage diferent from "...Amazon object-level immutability is more of a marketing term, in reality what they sell behind this term is regular object versioning..." ?

Is there any reason why WasabiDan does not answer this question?

[Gostev]

Because he did not log on here once after his only post above I just checked his forum account.

In any case, I can answer this: Wasabi does not support S3 Object Lock at all at this time.

I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - just requires a lot more code from our side to work with, for obvious reasons. I mean - a single, actually immutable copy of an object that cannot be deleted or overwritten would have been so much simpler to architect against. While dealing with potentially deletable and overwriteable "immutable" objects requires tracking the required object versions in its version history (separately for each and every object), which adds significant complexity.

But anyway, it did not stop us

[dw432948jkk]

Did I understand correctly that veeam is working to set up veeam-backups so that they can be copied (not moved) (with whatever other third party program) to the immutable storage of cloud backup providers like Azure, AWS, etc.? That would be great because, to my knowledge, no backup program can do this.

[Gostev]

Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!

[yasuda]

...I also wanted to clarify: I did not mean that Amazon's implementation is somehow invalid. It does the job - ...

Thanks for the clarification!

Is it also true Glacier or Deep Archive data would be protected, because there is a minimum duration data has to be left there, before it can be deleted? Assuming you are only concerned with recovering your most recent backups.

[Gostev]

My understanding is that Glacier has minimum duration charge, not minimum duration storage. Meaning, you can still delete data immediately after uploading, but you will be charged as if the data was stored there for a minimum required duration.

[dw432948jkk]

Peter, actually what you explained sounds like something that was always possible with Veeam... if you use an incremental backup mode with periodic synthetic or active fulls, you can certainly use any 3rd party program to copy those backups to object storage buckets with immutability enabled. Thanks!

The problem with this is the veeam backup chain metadata file.

Even with incremental backups, this file will always have the same name and will not be transferred by the programs that are supposed to do the uploads (rclone, cloudberry, etc.) because, for example, Azure reports back that the file already exists and can not be overwritten.

There is (unless you create a script that versioned this file before each upload and then always uploaded a modified version) from the perspective of upload programs no solution.

Is there a solution planned here from veeam?

[Gostev]

You might be confusing Veeam with some other vendor, because Veeam names each incremental backup file uniquely.

[dw432948jkk]

The files ending in .vib have different names, but one .vbm file always has the same name within a backup.

Well, I'm actually confused now

[Gostev]

VBM is a metadata file which is not essential for restore, as the actual data sits in VBK (full) and VIB (incremental) backup files.

Click Browse and select the necessary VBM or VBK file. If you select the VBM file, the import process will be notably faster. It is recommended that you select the VBK file only if the VBM file is not available.

[dw432948jkk]

Ie. the vbm file is actually solely responsible for enabling a faster backup?

Otherwise, there are no disadvantages if you do not have the vbm file?

So far it has not emerged for me from the documents.

[veremin]

It's not responsible for enabling faster backup, it makes backup import procedure work faster.

If you'd like to transfer backups offsite via 3-party utility, you can skip .vbm file. However, should disaster happen, you will need to import backup chain to backup server first (before you restore anything), without .vbm file this process will be significantly longer.

Hope it helps.

Thanks!

[BackupMonkey]

Hi all.

I'm just in the process of testing this and I'm completely confused.

Setup Azure Blob repository (AB1)
Setup local repository (LR1)
Setup Scale out repository to have LR1 and AB1 as members. Set it to move backups older than 0 days. (SOR1)
Setup job VM backup job (VMB1) to backup to SOR1
Setup a Backup Copy job (BCJ1) from VMB1 as source to SOR1 as target.

BCJ1 will fail with the error "Restore point is located in Backup Copy target repository and cannot be used as a source"
Assuming here that while Veeam understands tiers, backup copy jobs don't.

So,
Create another local repository (LR2)
Set VMB1 to use LR2
Set BCJ1 to use LR2 as source

BCJ1 will then copy the backups between the local repositories and yet again, nothing goes to azure. Grr


I can't backup direct to azure since I can't pick the AB1 as the primary destination for the backup jobs.
I can't do a backup copy job between tiers of the same scale out repository since the target and destinations have to be different and the job can't see the tiers.
I can't do a backup copy job between a local reposatory and a Azure repository because I can't select the azure repository.

No direct route & No backup copy route.

What am I missing here?

[wishr]

Hi,

You do not need copy jobs to offload your backups to Capacity Tier - it is done automatically using SOBR Offload Job, according to operational restore window.

This topic has been discussed many times - please review the documentation and posts above and let us know if you require additional information.

Thanks

[BackupMonkey]

Hi wishr.

I've seen that thread and can I point out.

Sorry to hear that, Phil. As I've said, this is already planned for the future update based on the feedback from beta - but I do understand that you may not be willing to wait. I hope you will consider Veeam again once this feature is delivered!

It's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.

The offload doesn't happen automatically for me.
Are you saying that I have to completely disable the job before the SOBR will run? because being active but not transfering data doesn't appear to be enough, and if that is the case how do you SOBR backup jobs that have SQL transactional logging enabled?

[wishr]

Capacity Tier was first introduced in B&R9.5 U4. The current version is 9.5U4b and the logic behind the purpose of this feature has not changed from that time.

To properly setup Capacity Tier I'd recommend referring to the Capacity Tier documentation, but in short words, the process looks like this:

1. You configure an Object Storage Repository;
2. Configure SOBR and add the aforementioned Object Storage Repository as Capacity Tier of SOBR;
3. Create a backup job targeting SOBR;
4. Offload job offloads created backups to Object Storage every 4 hours, according to the operational restore window you have configured. The same can be done manually when needed.

Thanks.

[Gostev]

It's been over a year since that was posted. As a question then, what version number update is Gostev talking about in this post? One would assume that a year is long enough to put something like this in place given that it's a selling point of the product and a highly requested feature.

I was talking about v10. Yes, the year will indeed be enough however, so far we're only into the 10th month since our previous release, whereas our major release cadence is annual.

[BackupMonkey]

@Wishr - I'm currently reading the Capacity Tier documentation again, I've clearly missed something. I can get the SOBR to run by hand just not automatically at this time. I'll track it down.
@Gostev - So next update then?

[wishr]

I'd like to add that SQL t-log backup (.VLB) is not a part of a backup chain, thus will not be offloaded to Object Storage.

[Gostev]

@BackupMonkey yes, this is a feature of v10, which we're planning to ship by the end of this year.

[pstilber]

As an update to my response on page 1, I'd be fine with the capacity tier actually following the "move backups after 0 days" setting. As it is now that setting is very misleading as the active chain and the previous two backups aren't offloaded. I can't be the only person who incorrectly assumed "move after 0 days" did not mean "or after 9 days, whatevs".

I'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well. Basically, for DR purposes I want all of my backups in Azure ASAP, while keeping a week or two on site for faster restores over our most commonly requested RPOs. I don't particularly care if this is accomplished with a backup copy job or inside of the SOBR tiering itself; though I'd prefer inside of the SOBR tiering for fewer jobs to maintain.

I tried using an Azure File Share as a target for a backup copy job and the performance of that configuration was only slightly faster than copying the 1s and 0s by hand.

[Gostev]

I'd ideally like a way to specify that all backups get copied to Azure immediately, while x days are kept on site as well.
I'd prefer inside of the SOBR tiering for fewer jobs to maintain.

The good news is, this is exactly how v10 works.

[vkky2k]

I understand I can configure AWS S3 or Azure Blob for one of Veeam repository. I probably should back up data to the local repository first, before tier the data to the cloud.

The question is, can I back up data directly to the cloud object storage, without staging in the local first?

Thanks!

[wishr]

Hi Roger,

I'm merging your topic with an existing thread - please take a look since it should answer all potential questions you may have in that regard.

Thanks

[vkky2k]

I am not so clear on what the answer is. Can Veeam allow me to back up data to Cloud repository without going to a local repository?

[Gostev]

No, you cannot backup directly to object storage.

[fernbrun]

Hello,

I have currently setup Veeam backup with 2 local repositories (1 for backup and 1 for archive job) and also a remote archive repository (in case of disaster in our DC).

So far everything works, but I would like to setup a 3 point of backup in the cloud, in case of major disaster in the town where is my company.

I already created an azure object storage repository and I would like to simply always add the last backup point + 7 last incremental backups points.

I understood that for that I need to use SOR, but based on what I read the backup are uploaded to the cloud during the offload job for archiving, but what I would like to achieve is to keep the data in local repository as well.

Could you please help let me know if this is doable and if how to configure it ?

Thanks

[wishr]

Hi Fernbrun,

"Copy" to object storage functionality is coming in B&R v10 which is expected to be released soon. With 9.5 Update 4b, you can only "move" aging backups to Azure.

Thanks