File Copy Job for configuration failing after upgrade to V13.0.1

[d.artzen]

Hello together,

I just wanted to ask if anyone else has issues with "File Copy Jobs" after upgrading from V12 to V13 on Windows. We use a "File Copy Job" to copy the Veeam configuration backup from the local disk to a hardened repository (since it is not possible to define two locations for the configuration backup). This worked fine up to and including V12.3.2 but since V13 it fails with a "Connection forwarder with ID {7c04b737-d250-4484-a1f4-e9c4eda6dc21} not registered Connection forwarder with ID {7c04b737-d250-4484-a1f4-e9c4eda6dc21} not registered" error. The hardened repo itself is correctly reachable, since our "Backup Copy Jobs" to that repository are all successful.
Support is still investigating (they can see that the VBR can connect, but the session ID created for the data mover agent in the Veeam server was not registered or closed), I just wanted to ask here if I am the only one with such an issue.

Case#: 07895143

Best regards
Daniel

[david.domask]

Hi Daniel,

No issues come to mind at first blush, so please continue with Support on this one.

Out of curiosity, is it a "classic" Hardened Repository from the previous Hardened Repository v2 ISO or did you upgrade to the v13 Hardened Repository from the Veeam Infrastructure Appliance?

[d.artzen]

Hi David,

the hardened repository is a manually installed linux server (Ubuntu 22.02 LTS), at the time we installed that server the Veeam ISO did not exist. And since all other Backup Copy Jobs work fine I don't think it something directly related to it. Especially since in V12 this worked without problems.

[david.domask]

Got it, figured as such but didn't see it confirmed on the case. Please continue with Support, let's see what the investigation turns up. I do advise at some point scheduling the upgrade to the Infrastructure Appliance deployed hardened repository.

[d.artzen]

That will not be possible with our current hardware. The Linux server doesn't have much internal Storage capacity, we had an older QNAP NAS Systems which we attached via iSCSI to the linux server. The connection is a direct network cable from a dedicated interface of the linux server to the NAS. So the NAS is not connected to our network and only reachable through the linux server.
But this config is not supported by the VIA, so we have to stay on the self build system until a hardware refresh.

[d.artzen]

So support was able to find the cause for the failure, but the reason is strange. The VBR server has two NICs, one is configured for the production network, where all components are located. Vcenter, Proxies and the Hardened Repository are all on the same subnet. The second NIC has a different IP (completely different subnet) and has no default gateway configured. This second NIC is connected and has a link, but there are currently no other devices on that network. There are no preferred network rules configured.
For whatever reason during the "File Copy Job" the VBR tries to reach the repository with the IP of that secondary NIC and not the one in the same subnet. The Backup Copy Jobs use the correct interface and IP. This does not make sense to me from a networking perspective. The solution was for me to disconnect that second NIC since it is currently not in use.
But I still don't understand why this happened. In V12 the constellation was identical and there it worked fine. Not sure if R&D wants to take a look at that behavior.

[david.domask]

Thanks for the update -- indeed seems a bit odd. As I see from the case, Support is continuing with their research, so please continue with Support, and they will contact RND if needed.

[d.artzen]

While support is still investigating this issue, I just ran into another issue where this behavior seems to be the cause. I have a second case open (07898151) concerning a significant increase in the duration of our backup jobs to local disk after the upgrade. We are talking about taking almost 2 hours instead of 20 minutes.
We had a remote session planned for today, but since I deactivated the second NIC yesterday, the backups this night were almost as fast as before. So the engineer will put that case on hold until the root cause is found.
Just to inform you guys that this goes a bit deeper than previously thought.

[d.artzen]

Support explained to me that in V13 there is a new approach to the connection between the agents, which became more secure but it seems that it can cause different network interface selection.
So the recommendation is to either have only one connected interface on the VBR or to define the preferred Network in the Network Traffic Rules.

We did both now, just to be sure and the file copy job works and the Backup Jobs have almost the same duration as before (only a few minutes longer than before, which is not a problem).

[RubinCompServ]

Thank you for that update @d.artzen! I have multiple NICs on all of my VBRs, so this would have been a big deal if we had upgraded!

[iDeNt_5]

Me too on some environments where I usually work... so good to know! Thanks Daniel