-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Backup Copy Jobs missing Data Transfer option ...
Hi everybody,
currently we are struggling with a BackupCopy job that should copy data from a central OnPrem backup repository to an Azure StorageContainer. No SOBR is involved currently. We copy VM backups from different site locations to a central backup repository and can use pre-configured WAN accelerators in the Data Transfer window to minimize data amount and to limit data transfer speed.
Why is this option (Data Transfer) missing, when an Azure StorageContainer is configured as a target in a BackupCopy job? We have a target gateway server involved and we have a source (OnPrem) and a target WAN accelerator (Azure) available for use, but it looks like the two WAN accelerators cannot be used in this scenario. How is limiting network traffic supposed to be done in this case, for which we configured Network Traffic Rules?
Any reply is very much appreciated.
Thanks
Didi7
currently we are struggling with a BackupCopy job that should copy data from a central OnPrem backup repository to an Azure StorageContainer. No SOBR is involved currently. We copy VM backups from different site locations to a central backup repository and can use pre-configured WAN accelerators in the Data Transfer window to minimize data amount and to limit data transfer speed.
Why is this option (Data Transfer) missing, when an Azure StorageContainer is configured as a target in a BackupCopy job? We have a target gateway server involved and we have a source (OnPrem) and a target WAN accelerator (Azure) available for use, but it looks like the two WAN accelerators cannot be used in this scenario. How is limiting network traffic supposed to be done in this case, for which we configured Network Traffic Rules?
Any reply is very much appreciated.
Thanks
Didi7
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hi Didi7,
I see the same behavior in my lab but not immediately aware of such a limitation. Let me check internally and will update the thread when I have more information.
I see the same behavior in my lab but not immediately aware of such a limitation. Let me check internally and will update the thread when I have more information.
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hi David,
thanks for letting me know. Would you say this is common behavior when using an Azure StorageContainer or might this be a bug?
Btw, I am talking about Veeam B&R 13.0.1.2067 (Windows) in case anybody wants to know
Regards,
Didi7
thanks for letting me know. Would you say this is common behavior when using an Azure StorageContainer or might this be a bug?
Btw, I am talking about Veeam B&R 13.0.1.2067 (Windows) in case anybody wants to know
Regards,
Didi7
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Just confirmed, with object storage repositories from Hyperscalers (AWS, Azure, Google), this is by design; WAN Accelerators have specific interactions with the target storage that require reading blocks fairly consistently from the target storage which can be quite costly (many GET requests) and introduces other performance considerations. So not a bug, by design.
If main goal is to limit the traffic, doesn't the built-in Internet rule assist here?
If main goal is to limit the traffic, doesn't the built-in Internet rule assist here?
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hi David, good to know.
So the conclusion therefore is, deployment of WAN-accelerators do not make sense, as they cannot be used with Hyperscaler target repositories (this is by design).
When configuring backup repositories based on Azure StorageContainers you have the option to use Direct "Connection Mode' or configure one or more Gateway servers.
In order to configure the right 'Network Traffic Rules', the question is ... who is speaking with whom?
If you configure a BackupCopy job for data to be copied from one OnPrem location to another OnPrem location and use a source and target WAN accelerator, the traffic on the WAN-connection runs through the source and target WAN accelerators, therefore traffic rules can be limited between the IP addresses of both WAN accelerators.
Now, if configuring a BackupCopy job for data to be copied from one OnPrem location to an Azure StorageContainer the question is who is responsible for the traffic between both backup repositories?
Is it ...
* (Source) the main backup server (as no Proxy and WAN accelerator is involved and the backup repository server only holds the data)
* (Target) the gateway server(s) that can be configured in 'Connection Mode'
or who is in the end speaking with whom?
As far as we understand, you can configure more than one Gateway server in 'Connection Mode' for redundancy purposes?
Regards,
Didi7
So the conclusion therefore is, deployment of WAN-accelerators do not make sense, as they cannot be used with Hyperscaler target repositories (this is by design).
When configuring backup repositories based on Azure StorageContainers you have the option to use Direct "Connection Mode' or configure one or more Gateway servers.
In order to configure the right 'Network Traffic Rules', the question is ... who is speaking with whom?
If you configure a BackupCopy job for data to be copied from one OnPrem location to another OnPrem location and use a source and target WAN accelerator, the traffic on the WAN-connection runs through the source and target WAN accelerators, therefore traffic rules can be limited between the IP addresses of both WAN accelerators.
Now, if configuring a BackupCopy job for data to be copied from one OnPrem location to an Azure StorageContainer the question is who is responsible for the traffic between both backup repositories?
Is it ...
* (Source) the main backup server (as no Proxy and WAN accelerator is involved and the backup repository server only holds the data)
* (Target) the gateway server(s) that can be configured in 'Connection Mode'
or who is in the end speaking with whom?
As far as we understand, you can configure more than one Gateway server in 'Connection Mode' for redundancy purposes?
Regards,
Didi7
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Correct, in this case use the Internet Rule if you need to throttle the connection.
For Internet rule however, it works by detecting if the target connection is NOT in the private IP classes; if we're targeting a public IP, then Internet rule applies.
This step controls how you connect to the target object storage repository, either Direct Connection or through a gateway, and it works as you suppose in your post. You can configure multiple gateways for redundancy purposes naturally.Now, if configuring a BackupCopy job for data to be copied from one OnPrem location to an Azure StorageContainer the question is who is responsible for the traffic between both backup repositories?
For Internet rule however, it works by detecting if the target connection is NOT in the private IP classes; if we're targeting a public IP, then Internet rule applies.
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Many thanks for replying my questions David, appreciated.
Our datacenter infrastructure is connected through an Express Route to Azure but why can't we throttle traffic by adding a Veeam Network Traffic Rule with the Veeam Gateway Server IP addresses in Azure and the main Veeam Software Appliance in our datacenter?
Thank you.
That is something though that really makes us a little bit skeptical. As we only use none public IPs to interact with the corresponding Azure StorageContainers, we wonder how we can throttle traffic in this case?For Internet rule however, it works by detecting if the target connection is NOT in the private IP classes; if we're targeting a public IP, then Internet rule applies.
Our datacenter infrastructure is connected through an Express Route to Azure but why can't we throttle traffic by adding a Veeam Network Traffic Rule with the Veeam Gateway Server IP addresses in Azure and the main Veeam Software Appliance in our datacenter?
Thank you.
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Got it, with PrivateLink it is a bit more complex, and regrettably you'll need to create rules for the IPs used by your Private Endpoints, and if I remember right these may rotate or be a rather large range, so it may be a few rules.
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hello David, in this case the OnPrem backup server is one IP address and the Gateway server in the cloud is another IP address, so the rule is pretty simple to build, as we discussed these are the components that talk to each other to transfer the backup data from an OnPrem backup repository to the Cloud storage container.
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Using the storage container in Cool-mode worked pretty fast, using another storage container in Hot-mode didn't work flawlessly and we had significant transfer problems with the following error ...
We wonder where's the problem now? What does the error tell us? Problems downloading the 'TESTSERVER-flat.vmdk' from the source or problems uploading the 'TESTSERVER-flat.vmdk' to the target?
We are not sure. Maybe someone else knows how to interpret this error?
In the meantime, we will transfer the relevant VIA-based Hardened Backup Repository (a VM) from one to another VMware cluster, where the source is a Nutanix cluster based on NFS storage and the target is an HPE cluster with dedicated FC-storage.
Thank you.
Hard disk 1 (50 GB)
Error: Unstable connection: unable to transmit data. Failed to upload disk 'TESTSERVER-flat.vmdk'
Exception from server: End of file [asio.misc:2] Unable to retrieve next block transmission command.
Number of already processed blocks: [3908]. Failed to download disk 'TESTSERVER-flat.vmdk'.
(Unstable connection: unable to transmit data.) (Failed to upload disk 'TESTSERVER-flat.vmdk')
(Exception from server: End of file [asio.misc:2]) (Unable to retrieve next block transmission command. Number of already processed blocks: [3908].) (Failed to download disk 'TESTSERVER-flat.vmdk'.)
Processing finished with errors at 5/8/2026 8:12:06 PM
We wonder where's the problem now? What does the error tell us? Problems downloading the 'TESTSERVER-flat.vmdk' from the source or problems uploading the 'TESTSERVER-flat.vmdk' to the target?
We are not sure. Maybe someone else knows how to interpret this error?
In the meantime, we will transfer the relevant VIA-based Hardened Backup Repository (a VM) from one to another VMware cluster, where the source is a Nutanix cluster based on NFS storage and the target is an HPE cluster with dedicated FC-storage.
Thank you.
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
https://www.veeam.com/kb2140Error: Unstable connection : unable to transmit data. Failed to upload disk 'TESTSERVER-flat.vmdk'
From experience, "unstable connection" means exactly like the KB article tells -- this error has some validation leading up to it to try it, meaning it's not triggered just by a flaky connection but instead the behavior experienced by the data mover agents suspects it is being blocked.
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hello David, all of the OnPrem components (main backup server and hardened repositories) work in a dedicated class C subnet, the components in Azure (Gateway server) also work in their own dedicate subnet. First I thought, the WAN-traffic is done through the main backup server and the Azure Gateway server, i.e. all backup copies from an OnPrem backup repository uses the way through the main backup server and the main backup server sends the data to the Azure Gateway server, that is configured for a specific Azure Storage Container, so that you must create an IP rule with the main backup server and the Azure Gateway server and that's it, but unfortunately the traffic is not limited.david.domask wrote: Apr 23, 2026 7:43 am Got it, with PrivateLink it is a bit more complex, and regrettably you'll need to create rules for the IPs used by your Private Endpoints, and if I remember right these may rotate or be a rather large range, so it may be a few rules.
I therefore decided to create an IP rule with the OnPrem dedicated Class C subnet, that means all 254 IP addresses and the dedicated Class C subnet in Azure, where the Gateway server is located and it seams as if the traffic is throttled now.
I assume for the Azure Storage Container, you can specifically mention the Azure Gateway server in the IP rule, as this is the only server component configured in the Azure Storage container, that is responsible for the backup data and that should be sufficient to limit or throttle the traffic.
Regarding the OnPrem components, I assume you need to mention more than just the main backup server in your IP rule, as it looks like that not only the main backup server talks to the Azure Gateway server, right?
Will have some more tests now to further limit the amount of IPs, that are necessary in the IP rule.
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
-
david.domask
- Product Manager
- Posts: 3715
- Liked: 905 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Hi Didi7,
Depends, if it's direct connection, it's from the Backup Repository(-ies) to your private endpoints, if you use gateways, you will need to include rules for the allowed gateways.
Depends, if it's direct connection, it's from the Backup Repository(-ies) to your private endpoints, if you use gateways, you will need to include rules for the allowed gateways.
David Domask | Product Management: Principal Analyst
-
Didi7
- Veteran
- Posts: 622
- Liked: 108 times
- Joined: Oct 17, 2014 8:09 am
- Location: Hypervisor
- Contact:
Re: Backup Copy Jobs missing Data Transfer option ...
Ok, let me clarify this again ... currently I am talking about a test environment with 2 Azure Storage Containers and in each Azure Storage Container there is the same Azure Gateway server configured (no direct connection).
Backup copies which are located in a Veeam hardened backup repository server being part of the above mentioned OnPrem Class C subnet, where also the main backup server appliance is located, now need to be transferred to the Azure Storage containers (with only one Azure Gateway server configured).
Rule before:
Source with main backup server IP address only
Target with Azure Gateway server IP address only
=> no throttling
Rule now:
Source with Class C subnet (where main backup server and hardened repository server are located)
Target with Class C subnet (where Azure Gateway server is located)
=> throttling active
My assumption, the target IP range can be limited to the Azure Gateway server(s), the source IP range must contain all IP addresses from all OnPrem components of the respective Class C subnet. It is not only the main backup server that transfers the data between the hardened backup repositories and the Azure Gateway servers, right? I thought, as you cannot use WAN accelerators in this scenario, that only the main backup server is involved in that transfer traffic on the OnPrem site.
P.S.: the production environment will have 2 Azure Gateway servers that are redundant, which means for the target IP address range, we will need to include the IP adresses for both Azure Gateway servers respectively
Backup copies which are located in a Veeam hardened backup repository server being part of the above mentioned OnPrem Class C subnet, where also the main backup server appliance is located, now need to be transferred to the Azure Storage containers (with only one Azure Gateway server configured).
Rule before:
Source with main backup server IP address only
Target with Azure Gateway server IP address only
=> no throttling
Rule now:
Source with Class C subnet (where main backup server and hardened repository server are located)
Target with Class C subnet (where Azure Gateway server is located)
=> throttling active
My assumption, the target IP range can be limited to the Azure Gateway server(s), the source IP range must contain all IP addresses from all OnPrem components of the respective Class C subnet. It is not only the main backup server that transfers the data between the hardened backup repositories and the Azure Gateway servers, right? I thought, as you cannot use WAN accelerators in this scenario, that only the main backup server is involved in that transfer traffic on the OnPrem site.
P.S.: the production environment will have 2 Azure Gateway servers that are redundant, which means for the target IP address range, we will need to include the IP adresses for both Azure Gateway servers respectively
Using the most recent Veeam B&R in many different environments now and counting!
*** Nominated for being the earliest early adopter of VSA ***
*** Nominated for being the earliest early adopter of VSA ***
Who is online
Users browsing this forum: AdsBot [Google], Amazon [Bot], MarkBoothmaa and 986 guests