Discussions related to using object storage as a backup target.
Post Reply
ajgball
Influencer
Posts: 11
Liked: never
Joined: Nov 05, 2009 12:18 pm
Full Name: AJ GB
Contact:

Azure Data Box Issue

Post by ajgball »

We have had an Azure Data Box for a week and have been gradually uploading 60TB (100 million files) from two separate veeam 10.5 servers.

However the offloads were getting slower over the week and eventually started timing out. Azure storage Explorer was also timing out.

I logged a call with Microsoft as I didn't believe Veeam was at fault and after a couple of calls they sent the log files off to engineering.

On the last call they told us that they had not seen our use case for Azure Databox before ? They said the box was optimised for writing files and not deleting files. Obviously Veeam is deleting the older .vib blocks. Essentially there is a database on the the Azure Databox that holds a list of all the files and it becomes fragmented when files are deleted.

It seems the Azure Databox does not work well with large amounts of files/data when you are also removing files.

Alex
HannesK
Product Manager
Posts: 14844
Liked: 3086 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Azure Data Box Issue

Post by HannesK »

Hello,
removal only happens if retention applies. So maybe you could extend retention to stop the deletes?

In general, it's known that Azure Data Box has limited scalability, because it's just one box instead of a classic scale-out system

Best regards,
Hannes
ajgball
Influencer
Posts: 11
Liked: never
Joined: Nov 05, 2009 12:18 pm
Full Name: AJ GB
Contact:

Re: Azure Data Box Issue

Post by ajgball »

Thanks Hannes, increasing the retention time may well be a good short term fix.
I was also considering :
1) copying the files to the Data Box in SMB format so there were a lot fewer of them.
2)Then Sending it back to Azure and spinning up a Veeam server in Azure. -
3)Uploading the backups to blob storage.
4) Finally enabling SOBR offload back on premise and letting it synch the changes.

Alex
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Azure Data Box Issue

Post by veremin »

Am I correct in assuming that you will be:

* deploying a backup repository in Azure for the restore points transferred over Azure Data Box
* adding it as a performance extent to scale-out backup repository
* adding object storage as capacity extent
* letting offload process copy restore points from performance to capacity extent
* switching performance extent to local repository

If so, the described scenario should work fine.

Thanks!
ajgball
Influencer
Posts: 11
Liked: never
Joined: Nov 05, 2009 12:18 pm
Full Name: AJ GB
Contact:

Re: Azure Data Box Issue

Post by ajgball »

Yes that's exactly what I was planning. Thankyou for confirming!

Alex
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Azure Data Box Issue

Post by veremin »

You're welcome. Let us know how the described process works for you and ask for additional help, if needed. Thanks!
ajgball
Influencer
Posts: 11
Liked: never
Joined: Nov 05, 2009 12:18 pm
Full Name: AJ GB
Contact:

Re: Azure Data Box Issue

Post by ajgball »

In the end I wiped the databox and started again with blob transfer but had extended the retention points on the backups as you suggested.
This worked well and we transferred 20TB in two days at roughly 1.5Gbs a second.
However having shipped the Databox back to Microsoft they are now uploading to the object store at only 1Gbs so that's another 2 days of wait time while my backups on-prem are incrementing all the time.
I had assumed that Microsoft would upload the data in a few hours at 10Gbs.
I'm learning that there are several gotchas with the Data Box.

Alex
Post Reply

Who is online

Users browsing this forum: No registered users and 7 guests