Copy or Move VBK to Azure WITHOUT using Object Storage

[arosas]

We have some older copy job files we no longer have a job tied to and they have the yearly individual VBK files. I would like to move these to Azure but still be able to restore and explore them through Veeam.
I will not use SOBR object storage due to limitation in object storage manageability of these files once moved and also not trusting object storage due to data loss we already had twice. (yes I have tickets on these issues)

I would like to configure an additional repository pointing to an Azure storage account and create a one time copy\move of these yearly VBK into Azure. Any ideas on how this can be accomplished?

[Mildur]

Hi Tony

I have some ideas how to build what you are looking for.
For example:
- You can use Azure Files over a vpn as an smb or nfs repo
- You can deploy a linux or windows vm in Azure to use it as a Backup Repo
With this both options, veeam can manage the vbk files and you will be able todo a restore.

I would‘t use Azure Files over the internet for backup files. But if it works for you one time each year, why not. I would prefer the option with the linux or windows vm in the cloud.

If you want to use object storage, there tools like azcopy where you can write your own script to upload the vbk files to a blob container. But this will not be integrated in veeam.

[arosas]

I was thinking Azure files with SMB or NFS as a backup repo since this option will let us still use Veeam explorer to open the VBK and we already have VPN to several regions in Azure.
Why are you saying you prefer using a VM in the cloud, what advantages\disadvantages does this have?

[Mildur]

Mounting an smb share from a remote location can lead to performance and stability issues. We are seeing that with Onpremise customers which are connecting from the gateway server to a smb share in another datacenter.

Best practice is to install a gateway server at the location of the smb share to overcome this possible connection issues.

If it works out for you, then I‘m happy for you.
If not, you can try to deploy a gateway server in the azure environment.

[soncscy]

Keep in mind, per Veeam's documentation, if you're not using their built in system, you're basically on your own. I don't recall if Azure has the same size limits Amazon does, but this is the biggest concern. Similarly, I wouldn't try to run a live restore off of such a connection, and instead would want to repatriate the data before trying. (As I get it, unless it's Capacity Tier, Veeam isn't "aware" of the blob storage/S3 nature and can't adjust to the API throttling)

So do this at your own risk. I get you're burnt by the built in capacity tier, but was it really a total loss? I have dozens of clients using capacity tier without incident; this doesn't remove the sting from your issues and I'm not trying to convince you otherwise, but maybe it was truly an exception?

As for using a VM in the cloud, it's just that it's far more predictable; you don't deal with the structure of capacity tier, and instead you just have a "normal" machine and "normal" files. The same protection the machine has, your files implicitly have.

[arosas]

@Mildur
Thank you for your response and i will investigate installing a gateway server in Azure and using that.

@soncscy
I would be using Veeam to do the copy\move and the repo configured in Veeam, so it would all be within the Veeam interface, nothing will be moved\copied outside the Veeam interface so i think we are covered there for support. I can see the stability issues and performance issues we may run into without using the Azure gateway server that was suggested, so I will investigate that portion.

As for the being burnt, yes, it was a very unfortunate scenario, the first time completely losing a little over 40TB of already tiered data using the SOBR object storage to Azure after an upgrade. It has been validated by escalation engineers on your end.
Just recently we have had a separate issue where the system thinks the data was offloaded but it really never was and it cannot be found on prem or in Azure.
Object storage also lacks the ability to granularly manage individual files that were tiered to Azure without bringing the entire data back down, which in our case some are huge 20+TB files.
Instead of dealing with object storage we will forgo using it for now.

Veeam does an excellent job of improving features over time and I'm sure object storage is no different, but until then I need to find a manual solution.

Thanks for the feedback, it's very much appreciated.

[soncscy]

> I would be using Veeam to do the copy\move and the repo configured in Veeam, so it would all be within the Veeam interface, nothing will be moved\copied outside the Veeam interface so i think we are covered there for support.

Just to be clear Tony, not quite:

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

> Object storage gateway appliances that are used to store backup data in filer (CIFS/NFS) or block device mode (iSCSI/FC/SAS) are not supported if the backup data is offloaded to object storage and is no longer stored directly on the appliance.
Such gateway appliances are only supported in the following cases:

All of the backup data is stored on the appliance altogether (that is, all of the backup chains are stored on the appliance as a whole and not scattered across multiple devices) and only additional copies of the backup data are transported to object storage.
These appliances emulate a tape system (VTL) as an access protocol for Veeam Backup & Replication.

Basically, Veeam doesn't support putting data on Azure storage except via capacity tier. Azure Files is a little different than Azure blob, but frankly speaking I don't see a difference. Maybe a Veeam employee can comment on restores directly from Azure Files. They had a special patch to give File Share support to Azure Files, so I kind of get the impression it's not the same.


> Object storage also lacks the ability to granularly manage individual files that were tiered to Azure without bringing the entire data back down, which in our case some are huge 20+TB files.

Sorry, what do you mean by this? I'm not here just to champion Veeam, but plenty of my clients are all in with Azure and we (they) do restores directly from Azure. What part required you to download 20 TiB? Except for entire VM restore to on-premises which logically should move all data, we've never needed to download entire backups to do restores.

(Side note: I suppose because of the green you think Fabian and I are Veeam employees; we're not, we're just active on the forums. I don't know Fabian's full background but I suspect it's similar to me, as I manage a moderately successful MSP and one of our offerings is Veeam. I don't have visibility into your cases with Veeam. But I get the confusion, the colors are fairly close.)

[Mildur]

I don't know Fabian's full background but I suspect it's similar to me, as I manage a moderately successful MSP and one of our offerings is Veeam. I don't have visibility into your cases with Veeam. But I get the confusion, the colors are fairly close.

Same for me

[arosas]

Thanks for clarifying. I will check with Veeam support on the design and be sure it's within a supported config, regardless of which way I decide to go.

As for the management issue.
If SOBR tiers older copy job VBK files like I have, which no longer have a backup job or copy job tied to them, let's say I have 10 files and all 10 were tiered to Azure. I can restore from them and even use veeam explorer which is great, but since they are no longer tied to a job, (older copy jobs no longer used but needed for retention) there is no retention managed by Veeam on those files. You can't simply delete those individual VBKs once they are offloaded, you can't just tell Veeam to delete say 3 out of the 10 offloaded VBK's, instead you must recall the entire VBK (20+ TB X 3) and delete it locally. Support already confirmed this portion after reviewing our environment, they currently lack the ability to manage individual VBKs files once they are tiered to Azure.

Regardless if you are Veeam support or not, you guys gave very good info to think about and I thank you for it.

[Mildur]

Your welcome

[soncscy]

Glad we could help!

As for your issue, now I get your idea -- are these full backups GFS ones though maybe? I was under the impression that this should be controlled by "Orphaned Backups" and their background retention, but if they're not GFS, I suppose that the GFS background retention won't help you there.

Out of curiosity, what does the storage cost for your particular agreement with Azure? I.e., how nasty is just letting it sit until your recovery period needs are done?

[arosas]

If my math is correct, it appears to be roughly 5k a year for 50TB that we have stored for this particular example using Azure cost management. However I do not pay the bill and I know it's significantly (by a lot) less according to our actual bill and our enterprise agreement, so I'm not sure what the actual price is.

[soncscy]

Yikes, that's quite the price to be honest.

Can you help me understand this though?

https://www.google.com/search?hl=en&q=. ... 0*%2051200
https://azure.microsoft.com/en-us/prici ... dn=disable

Even for the most expensive price here, .018 USD ,it's still < $1k USD: https://www.google.com/search?q=.018+USD+*+51200

This is a non-trivial amount of money, but it's not insurmountable as I see it. Can you share how you got to 5k (presumably USD)?

(Note: I convert everything to TiB/GiB, so I did 50 TB to GiB to get 51200 GiB. As I see it, 5k is a bit high...)