Azure Data Box Issue

[ajgball]

We have had an Azure Data Box for a week and have been gradually uploading 60TB (100 million files) from two separate veeam 10.5 servers.

However the offloads were getting slower over the week and eventually started timing out. Azure storage Explorer was also timing out.

I logged a call with Microsoft as I didn't believe Veeam was at fault and after a couple of calls they sent the log files off to engineering.

On the last call they told us that they had not seen our use case for Azure Databox before ? They said the box was optimised for writing files and not deleting files. Obviously Veeam is deleting the older .vib blocks. Essentially there is a database on the the Azure Databox that holds a list of all the files and it becomes fragmented when files are deleted.

It seems the Azure Databox does not work well with large amounts of files/data when you are also removing files.

Alex

[HannesK]

Hello,
removal only happens if retention applies. So maybe you could extend retention to stop the deletes?

In general, it's known that Azure Data Box has limited scalability, because it's just one box instead of a classic scale-out system

Best regards,
Hannes

[ajgball]

Thanks Hannes, increasing the retention time may well be a good short term fix.
I was also considering :
1) copying the files to the Data Box in SMB format so there were a lot fewer of them.
2)Then Sending it back to Azure and spinning up a Veeam server in Azure. -
3)Uploading the backups to blob storage.
4) Finally enabling SOBR offload back on premise and letting it synch the changes.

Alex

[veremin]

Am I correct in assuming that you will be:

* deploying a backup repository in Azure for the restore points transferred over Azure Data Box
* adding it as a performance extent to scale-out backup repository
* adding object storage as capacity extent
* letting offload process copy restore points from performance to capacity extent
* switching performance extent to local repository

If so, the described scenario should work fine.

Thanks!

[ajgball]

Yes that's exactly what I was planning. Thankyou for confirming!

Alex

[veremin]

You're welcome. Let us know how the described process works for you and ask for additional help, if needed. Thanks!

[ajgball]

In the end I wiped the databox and started again with blob transfer but had extended the retention points on the backups as you suggested.
This worked well and we transferred 20TB in two days at roughly 1.5Gbs a second.
However having shipped the Databox back to Microsoft they are now uploading to the object store at only 1Gbs so that's another 2 days of wait time while my backups on-prem are incrementing all the time.
I had assumed that Microsoft would upload the data in a few hours at 10Gbs.
I'm learning that there are several gotchas with the Data Box.

Alex