SOBR Offload Job very slow

[tjdoc]

Hi,
I am working on a Veeam poc and have configured a SOBR to offload a copy of a backup to an Azure Blob storage container.
The VBR server is also being used in this case as the Performance tier to store the local backup.
The test VM produced a full backup file of 68GB in size on its first run, this completed fairly quickly in about 6 mins.
The SOBR offload job then started and from the log, it reads the .vbk file initially.
Unfortunately when running this initial read, the "processing rate" was only 1MB/s - so processing the full backup file was extremely slow.
After about 30 minutes, only 2.5GB of 68GB had been read.

As this is just a test it's fine, but I'm worried that there is something preventing the speed from running faster which might also find its way into our real environment once we set it up.
We have much bigger "real" backups which means there is no way they would complete in the backup window.

I have previously tested backing up the same server multiple times, but instead of using a SOBR to copy up to Azure, I configured a Backup Copy job - this just copied the entire file in about 20 minutes or so, no issues really.
There seems to be a huge difference between using the SOBR offload and a basic backup copy job.

Please could anyone suggest why we might be seeing this problem and what we can do to improve the performance?

Thanks very much
Tim

[david.domask]

Hi Tim,

Thank you for the description of your testing so far; may I ask, from the offload session window, what was listed as the bottleneck? Source by chance? That means typically that it's the Data Transfer step here which is likely bottlenecking the offload which would put the focus on the Performance Tier extents or possibly the gateway.

I'm afraid it will be difficult to troubleshoot specifically over the forums, so please open a Support Case as is noted when creating a topic and let's allow Support to review the situation. Remember to include logs for Support to review. (Please use the 3rd radio option and select the Veeam Server itself to export logs which include the SOBR offload logs. Within your case please specify the name of the SOBR to focus on for the review).

Share your case number here once opened, thanks!

[tjdoc]

Hi David,
At the moment I am only using a trial version of VBR to test things out, so I'm assuming that I am not able to log a support ticket without a valid contract / licence?

In case it helps, I can see the following info in the statistics window:

Processing rate 1MB/s
Bottleneck: Target

These log entries seems to show the problem:
19/07/2024 02:01:05 :: Preparing offloaded backups...
19/07/2024 02:01:12 :: Queued for processing at 19/07/2024 02:01:12
19/07/2024 02:01:12 :: Required backup infrastructure resources have been assigned
19/07/2024 02:01:24 :: server1.eea59c52-6257-413d-a2dc-a3e7b9beD2024-07-17T182013_251C.vbk (67.7 GB) 15.5 GB read at 1 MB/s --note the duration of this was 3 hours 27 mins.
19/07/2024 05:43:15 :: Busy: Source 0% > Proxy 7% > Network 0% > Target 98%
19/07/2024 05:43:15 :: Primary bottleneck: Target

I can't really see any restrictions on transfer rate on the Azure account, but in any case if I create a backup copy job the transfer happens quickly, so I don't believe there's a rate cap on 1mb/s.

In the config of the local backup repo, the number of concurrent tasks is limited to 4. No setting enabled to limit read and write data. For the Azure repo, I have not ticked the setting to limit concurrent tasks. Do these settings sound ok?


Any ideas based on this at all?

Thank you
Tim