Switching between object storage repositories

[pstilber]

We currently use Azure for the capacity tier in our SOBR and we want to switch to Backblaze for immutability. Is there a way I can switch back to Azure if we experience issues with Backblaze without losing our restore points? There is a warning about losing access to those restore points when switching which repository is used for the capacity tier and I want to be sure that just means "while Azure isn't the capacity tier" and not "forever".

[dalbertson]

My recommendation would be to start a new sobr and test our backblaze prior to moving anything.

Pulling data out of azure is an expensive process and you only want to do it if you have to.

If you decide to go with backblaze you can then either pull the data back local and then send to backblaze or you can let them age out on azure and send anything new to backblaze.

[pstilber]

Am I not able to switch between the two capacity tiers without losing data in the same SOBR?

[Gostev]

You can, by using the pull method: download backups to the Performance Tier, switch the Capacity Tier to use new object storage, and have SOBR offload backups to it again.

[dalbertson]

Correct, as Gostev and I mentioned you can pull the data and send it to a new capacity tier.

But again I want to remind you that there are costs associated with pulling data out of azure. Just keep that in mind and design this out to see what the most cost effective solution would be for you.

[pstilber]

Thanks! I was hoping there was a sort of "maintenance mode" I could set to suspend the repository so I could test out a different provider without having to tear everything down and build it back up again, but it appears that isn't the case.

[dalbertson]

Sure.

Here is the guide on migrating the data
https://helpcenter.veeam.com/docs/backu ... ml?ver=110

You can put a capacity tier into seal mode which is described here
https://helpcenter.veeam.com/docs/backu ... ml?ver=110

[pstilber]

Can I seal the current Azure capacity tier repository and then switch the SOBR to use the new Backblaze repository without losing the ability to restore from the Azure repository? I'm fine with starting a new backup chain with an active full to populate the Backblaze repository instead of exporting my data from Azure; in fact that's what I would prefer.

[dalbertson]

You can do that. You will add backblaze and then you will need to seal the azure repo.

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

[pstilber]

FYI, your instructions and the helpcenter link are incorrect. If you don't seal the repo first it disappears from both the SOBR and the Repositories view in Backup Infrastructure when you switch to the other repository for your capacity tier leaving you with no way to restore, or even see, the previous capacity tier repository. Luckily this is only going to lose a week of backups for me, but others may not have most of their backups kept locally as well like I did. I know the backups are still there because if I try to switch back to that repository it tells me backups exist there and need to be imported first, but I can't interact with them in any other way with Veeam.
I'm not sure if the instructions were always incorrect, if v11 changed something, or if something about my setup is different; but I figured you folks would like to know before others follow those instructions and lose backups and a weekend like I did.

[Gostev]

leaving you with no way to restore, or even see, the previous capacity tier repository

But this exactly what the warning message box says on the screenshot following the link Dustin posted?

[pstilber]

Can I seal the current Azure capacity tier repository and then switch the SOBR to use the new Backblaze repository without losing the ability to restore from the Azure repository? I'm fine with starting a new backup chain with an active full to populate the Backblaze repository instead of exporting my data from Azure; in fact that's what I would prefer.

You can do that. You will add backblaze and then you will need to seal the azure repo.

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Do you see why I might have been confused? I knew that removing the repository would render it unusable, that is why I created this thread asking this exact question. Go back and look at the first post if you don't believe me.

[Gostev]

Totally, actually my message was more for @dalbertson to explain what he meant by "you can do that".

And also to say that the help center link content seems correct, as it also explicitly explains that "you will not be able to restore your data from it".

[dalbertson]

Hi @pstilber Forgive me for the misunderstanding then. What i meant by my response is that Yes you can seal the extent and remove it from a SOBR and the data will still remain on the sealed extent. In some of the previous replies i thought it was understood that you could only have one object extent active in the SOBR at a time.

The use case i envisioned was that you wanted to leave the data on the azure extent and then start new with BB. If for any reason you needed the azure repo data you could still restore from it. The "old" azure repo should still be under backup repositories correct.

[Gostev]

I don't think sealing makes any difference Dustin? You just remove the extent from SOBR Capacity Tier and that's it. All backups will stay there regardless - and if you ever need to restore from them, you just import them with the Import Backup functionality.

[dalbertson]

Correct, When it was mentioned that there was a want for a "maintenance mode to suspend the repo" , thats why Sealing the extent was mentioned.

There will be more clarity next time.

[pirx]

I don't think sealing makes any difference Dustin? You just remove the extent from SOBR Capacity Tier and that's it. All backups will stay there regardless - and if you ever need to restore from them, you just import them with the Import Backup functionality.

Exactly this is not working for me. See 04618351. According to support there is no easy way to access old backups. Did anybody test this before?

[Gostev]

Of course... in fact, this was the main demo at the V10 launch!

However, I understand your problem is you're trying to import them into the same backup server - which is never the case when performing a DR restore like the one we demoed, as the original backup server is gone. And I can see why this does not work due to conflicting backup IDs.

So, importing those backups into a separate backup server instance and performing restores from there (if you ever need to) is the way to go.

In general, expectations for changing object storage repositories is that users will do Download + Offload again. Can it be made easier in future? Sure, however it's not exactly a common need to switch between object storage repositories back and forth, so it has never been a priority use case for us so far.

[pirx]

Importing backups into a separate backup server is not a real world option. We have an operations team that performs FLR or even VM restores each day. Now we have to build up a second Veeam environment + proxies etc, access to vSphere, open firewall and tell those people they have to search in two environments for backups? This is simply not acceptable (and I don't talk only about me).

It's somehow amazing that nobody thought about this. I mean it's not that we want to do this. It's 1.) because we hit a bug and Veeam DB <-> S3 is still out of sync 2.) we used one bucket with multiple folders for several buckets, which was our fault and we need to change. But this can happen for a lot reasons. Even support told me right at the beginning that it's not unusual that buckets have to be replaced with new ones because of different kind of problems. This means every customer that has to do this is struggling to access existing backup that were offloaded.

[Gostev]

It's somehow amazing that nobody thought about this. I mean it's not that we want to do this.

This only proves my point though... it's a rare scenario and even you did not want to do this, rather circumstances have forced you to. So how could I possible prioritize this scenario for our R&D, when there are dozens of scenarios that need improvement which many more customers are facing much more often. I know this does not help you, I merely try to explain use cases prioritization.

Also, it's not that "nobody thought about this" in general. It's the opposite, there's actually the fully documented and fully supported approach of object storage repository migration.

[pirx]

In any case this is something that one should be aware of, before moving data to S3 buckets. Now we have to see how we proceed, I simply see no good option with 400-500TB in S3.

[Gostev]

One thing I've been told just now is that our support engineers came up with the instructions on how to cleanly migrate from your situation (one bucket shared among multiple SOBRs) to the recommended deployment (dedicated bucket for each SOBR) without performing full data retrieval (which saves money) and while preserving access to "old" backups. So if you don't rush and stick to their instructions closely, looks like there's a chance this will still end well... to the most part

[pirx]

Yes, there is still the downside that we have to stop and disable affected jobs. Then do some steps (remove jobs from configuration...), import backups from S3 and.... wait. This can take some time for S3. And we will not have backups as no job is running. And all of this sounds very... exciting. I just didn't expect this as I was told here post394151.html#p394151 (I think I explained exactly this use case) and also from support beforehand that I'd be able to import existing backups. This is something I planned for 3 months now and nobody told me before removing the first bucket that _this_ is necessary. Thus my question if this was ever tested. Anyhow, I just hope nobody will need a restore from the offloaded backups from that first bucket that was removed.