VEB -> VBR host Nic selection

[aaron@ARB]

I have VEB pointing at my VBR repository. the VBR host has 3 IP's. However VEB can only route to one of them (its internal facing one), but im guessing that the VBR host is telling VEB of all its IP's so every time it needs to talk to it, VEB tries all 3 IP's which slow things down as you can see from the below

Code: Select all

.04.2016 15:36:53] <01> Error    Failed to connect to agent's endpoint '10.1.1.110:2500'. Host: 'ARBBackup.ensat.local'.
[12.04.2016 15:36:53] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.1.110:2500 (System.Net.Sockets.SocketException)
[12.04.2016 15:36:53] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[12.04.2016 15:36:53] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[12.04.2016 15:36:53] <01> Error       at Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[12.04.2016 15:36:53] <01> Info     [NetSocket] Connect
[12.04.2016 15:37:14] <01> Error    Failed to connect to agent's endpoint '10.1.2.190:2500'. Host: 'ARBBackup.xxx.xxxl'.
[12.04.2016 15:37:14] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.2.190:2500 (System.Net.Sockets.SocketException)
[12.04.2016 15:37:14] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[12.04.2016 15:37:14] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[12.04.2016 15:37:14] <01> Error       at Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[12.04.2016 15:37:14] <01> Info     [NetSocket] Connect

The VEB host is out on the WAN and wont be able to talk to either of the othe 2 addresses it is trying (10.1.2.190 and 10.1.1.110). Is there a way to force VEB to just use a particular IP when communicating with the VBR host?

Cheers.

[PTide]

Hi,

Routing is performed on the OS level, not by VEB, so currently there is no way how you could force VEB to use some particular interface.

Thank you.

[aaron@ARB]

Hi,

Okay, but then can VEB just attempt to use the IP that its been given in the configuration? Does it really need to enumerate all the IP's from the VBR host? I've given it the IP that I want it to use, I don't need it to try and connect on the other 2 that i didn't ask it for?

In my example, I configured the actual backup job with 192.168.1.190, i didn't make any mention of the other 2.

Do you see what I mean?

[aaron@ARB]

I tested this again by disabling the NIC with the other additional IP's and they still show up in the log file but VEB does not try and connect to them

Code: Select all

[13.04.2016 13:09:19] <01> Info     [ReconnectableSessionStarter] Creating reconnectable socket with options: [enabled: timeout 1800000, interval 15000]. Agent on [192.168.1.190:2501;10.1.1.110:2501;10.1.2.190:2501]. Connection id: b6b626a9-89e0-403f-9a40-02ebb70c810a.z

however as soon as I re-enable the NIC it tries again

Code: Select all

[01.04.2016 19:31:24] <01> Error    Failed to connect to agent's endpoint '10.1.1.110:2530'. Host: 'ARBBACKUP'.
[01.04.2016 19:31:24] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.1.110:2530 (System.Net.Sockets.SocketException)
[01.04.2016 19:31:24] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[01.04.2016 19:31:24] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[01.04.2016 19:31:24] <01> Error       at Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[01.04.2016 19:31:24] <01> Info     [NetSocket] Connect
[01.04.2016 19:31:45] <01> Error    Failed to connect to agent's endpoint '10.1.2.190:2530'. Host: 'ARBBACKUP'.
[01.04.2016 19:31:45] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.2.190:2530 (System.Net.Sockets.SocketException)
[01.04.2016 19:31:45] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[01.04.2016 19:31:45] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[01.04.2016 19:31:45] <01> Error       at Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[01.04.2016 19:31:45] <01> Info     [NetSocket] Connect
[01.04.2016 19:31:45] <01> Info     [ReconnectableSessionStarter] Creating reconnectable socket with options: [enabled: timeout 1800000, interval 15000]. Agent on [10.1.1.110:2530;10.1.2.190:2530;192.168.1.190:2530]. Connection id: cf3b6c8f-7fdc-4d43-b9dd-645f7a4453ee.
[01.04.2016 19:31:45] <10> Info           [AP] (a37c) output: >
[01.04.2016 19:31:45] <01> Info                   [AP] (a37c) state: ready
[01.04.2016 19:31:45] <01> Info     [ProxyAgent] Starting client agent session, id '26b22f28-0ec1-4bc6-8c0a-d308c10a0350', host 'ARBBACKUP', agent id '61ebb77c-02cf-42f1-9107-f5b04853033f', IPs '10.1.1.110:2530;10.1.2.190:2530;192.168.1.190:2530', PID '24380'
[01.04.2016 19:31:45] <01> Info     [SocketAgentService] Connecting to agent 'ARBBACKUP' ('10.1.1.110:2530;10.1.2.190:2530;192.168.1.190:2530')
[01.04.2016 19:31:45] <01> Info     [NetSocket] Connect
[01.04.2016 19:32:06] <01> Error    Failed to connect to agent's endpoint '10.1.1.110:2530'. Host: 'ARBBACKUP'.
[01.04.2016 19:32:06] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.1.110:2530 (System.Net.Sockets.SocketException)
[01.04.2016 19:32:06] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[01.04.2016 19:32:06] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[01.04.2016 19:32:06] <01> Error       at Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[01.04.2016 19:32:06] <01> Info     [NetSocket] Connect
[01.04.2016 19:32:27] <01> Error    Failed to connect to agent's endpoint '10.1.2.190:2530'. Host: 'ARBBACKUP'.
[01.04.2016 19:32:27] <01> Error    A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.2.190:2530 (System.Net.Sockets.SocketException)
[01.04.2016 19:32:27] <01> Error       at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[01.04.2016 19:32:27] <01> Error       at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[01.04.2016 19:32:27] <01> Error       at Veeam.Backup.

Removing the IP's from DNS does not make a difference either, so I suspect its getting the IP's from the VBR host. Disabling those IP's is not an option.

[PTide]

In my example, I configured the actual backup job with 192.168.1.190, i didn't make any mention of the other 2.

That IP is used to establish a connection with VBR server. When Endpoint needs to connect to a repository it uses that IP/hostname to request VBR server to spin-up an agent on target and return IP address and port. If the repo is a shared folder then target agent is started on a proxy (a server that has a direct access to the share). It seems that in your case the repository is either a shared folder that is connected to VBR or VBR's local disk - in both cases target agent is started on VBR and has no information what IP did the request come from thus it passes all IPs that are available.

<...>so I suspect its getting the IP's from the VBR host.

Correct. Unfortunatley that behaviour cannot be changed currently but we will take a closer look into the situation to see what we can do.

Thank you.

[aaron@ARB]

Is there a registry setting that you can set to possibly reduce the timeout? at the moment its set for 18 seconds

Code: Select all

[13.04.2016 13:09:19] <01> Info     [ReconnectableSessionStarter] Creating reconnectable socket with options: [enabled: timeout 1800000, interval 15000]. Agent on [192.168.1.190:2501;10.1.1.110:2501;10.1.2.190:2501]. Connection id: b6b626a9-89e0-403f-9a40-02ebb70c810a.z

Is there a registry setting (hidden or otherwise) to lower that possibly to something more respectable? The latency on the WAN we have is measured in the ms, so even setting it to 2000-3000ms would be sufficient i would think

[PTide]

Is there a registry setting that you can set to possibly reduce the timeout? at the moment its set for 18 seconds

That timeout is actually 30 minutes (1800000ms) and has nothing to do with the connection timeout.

Also this part

A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.1.2.190:2530 (System.Net.Sockets.SocketException)

makes me think that not only VEB defines the timeout, but also an operating system (TCP/IP stack).

Anyway currently there is no registry key that can be used to reduce the connection timeout on the VEB side. If it turns out that there is a way to do that I'll update this topic.

Thank you.

[kickapoo]

Hi,
We're presently testiing a VBR server with the goal of deserving two differents network.
The server has two networks cards; one for each network as well.
The server is able to ping on the two networks without any problem.

Our problem is that when we use the endpoint or the Windows agent, we are not able to take the backup depending of the network card priority on the VBR server.
So when the priority is on the card X, I'm able to take backups of agent on the network X only and viceversa; otherwise the endpoint fails after 10 minutes trying to connect.
I also got to say the the client and the server are each able to ping each other at all times.

Thanks for your help,

[DGrinev]

Hi and welcome to the community!

Since routing is performed not by VEB but the OS, there is no way to force VEB to use preferred interface.
However, you can try to add the backup server in VEB by IP according to the desired network.

Please review this existing thread for additional information.

Thanks!

[kickapoo]

Hi,

FYI, a static route on the VBR server resolve our problem.

Thanks

[DGrinev]

I'm glad you solved the issue and thank you for sharing a possible workaround for future readers!

[Mattis]

Hello Veeam community,

We have a Backup Server with multiple Backup Network interfaces (~30 Networks tagged configured) and the clients are located on the backup networks.
Installed is Veeam B&R 9.5 Update 3 and the newest VAW agent is deployed (central managed).

I can connect and configure the clients without any issues but as soon as a backup is starting, the agent is getting a list with all network interfaces (data mover IPs) from backup server and it takes horrible long till he has founded the corrected NIC.

Setup preffered networks are not helping because it is only possible global in veeam.

In the logs I can find "Failed to connect to server 'backupserver'", addrs [IP:xx.xx.xx.xx, port:2500];..." and the last entry is "[IP:client-fqdn, port:2500]'"

Is it possible to force that the agent is connecting back to the same IP from Backup Server how the backup Server is polling / rollout and managing the agent?
If not, is there a possibility to force that at first the agent should try to connect to the backup server via FQDN and if the fqdn is not answering trying the provided data mover IPs?

PS: Support ID: #00966114


Many Thanks for your help,
Mattis

[ex-it]

Hi,

we got the same issue with multiple NICs on the Backup Server. I tried to connect a Linux Agent to the Backup Server from an external network with an NAT Rule on the firewall. vbm file will be created but when the agent try to connect to the dynamic ports (2500-5000) same error as Mattis mentioned above happend. Anyone has an answer or workaround to handle this issue?

[PTide]

Hi,

vbm file will be created but when the agent try to connect to the dynamic ports (2500-5000) same error as Mattis mentioned above happend.

Can you elaborate on this one please? Did you mean that the agent successfully creates .vbm file, however it fails to proceed the session as it cannot connect to the repository IP address on port 2500 despite the fact that the required port range is opened and NATed, is that correct?

Thanks

[ex-it]

Hi PTide,

exactly! .vbm File is created on the correct repository. But after snapshotting and trying to copy the vbk file it fails to connect. NAT Rule is the same rule as for Port 10006.

Btw, i've tried to connect the Linux Agent via SMB Mount with the same NAT Rule and this is working fine, but i have to import the backups manually to VBR to see them. And i only can create a filecopy Job for second side backup, which is pretty bad because of the backup chain.

[Yves_]

Hi there,

First of all amazing (and in my opinion BEST) backup solution you created there. Easy to use, fast and very very big feature set. With the new Veeam Agent for Windows which is able to backup to local storage / shared folder / veeam backup repository this takes a whole new dimension!

I am currently running a trial license to test out this new bad boy (Veeam Backup & Replication 9.5 Update 3) and installed it on a test Server (Win2k16Std) and installed the Agent on a test client (Win2k11SBS). The Win2k11SBS is not in the same network as the Veeam Backup Repository. So I natted 10001, 2500-5000 and Veeam Agent finds the Veeam backup repository... but as far as I see with resmon on windows it then tries to access the LAN IP (10.10.10.x) of the Backup Repository not the WAN IP (212.x.x.x) and fails.

What did I do wrong?

Thanks for your help

Best Regards,
Yves

[PTide]

Sending backups to the repo located behind NAT is not supported. On the other hand, if all ports are opened and both repo and VBR reside on the same host, it should work. That fact that it creates .vbm file successfully but cannot proceed with the backup is rather strange, so please contact our support for investigation.

Thanks

[ex-it]

Thanks PTide, i will do so.

[ex-it]

Hi everyone,

as thoght, support said that VBR behind NAT is not supported.

[JTT]

Hi,

FYI, a static route on the VBR server resolve our problem.

Thanks

Please share the solution?