Gateway Selection

[Dark-Sider]

Hi,

with a kind of small setup (5 ESXi Hosts, 5 Bare Metal Windows machines, 2 NAS for Repositories and a Tape Server with an attached Tape Library) I was tracking down some severe performance issues when writing a backup from our archive NAS to Tape. The Speed was capping out at 50-60MB/s which is a bit on the slow side for LTO-9

Only when I looked at the host-list within "resmon" I saw that the tape server was receiving all the data from one of our bare metal windows hosts - in this case our active-directory #4 server. Since that server only has a Gigabit NIC it caps out a ~600 Mbit/s in both directions.

I only learned at this point, that there exists something called a gateway when working with backup shares. Due to the nature of tape jobs they usually need a Gateway to produce the synthetic full for the tape job.

I wonder why veeam chooses that ActiveDirectory#4 Server. I checked the Gateway list within the Repository Entry. It was not not set to automatic, but to "manual" with all listed hosts "checked" this also includes ActiveDirectory#4, our main veeam-server as well as the tape server.

While our Backup-machines are not AD-joined but are on the same server-subnet as the NAS and all other Server I wonder why veeam chose that ActiveDirectory#4 with only 1 Gbit/s connection speed over the Tape Server which has plenty of local memory / cpu and 10 Gbit/s nic or at least the default veeam server which also is very beefy in regards of CPU / RAM and NIC?

regards,
Fabian

[DChiavari]

Hello, as far as I know the selection algorithm does not consider the nominal NIC(s) speed - it's mostly based on current CPU/RAM load, available task slots and network "vicinity" to the target (being in the same subnet). Maybe in your case the AD server has the least load of all when the jobs start? A potential suggestion could be removing (unchecking) that server from the list, as you're not using full automatic selection anyway.

[Dark-Sider]

Hi,

thanks for the reply. Yeah our ADs pretty much have 0 load (although not being over potent in the first place). I already did remove them from the gateway selection in the repository entries. Which should solve the issue on the next run.

What I find a bit distirubing is, that the gateways seem to use the UNC paths of the repository to read / work with the repository files. IT security best practice rules usually suggest to detach your backup environment so threat actors who might gain AD access have no access to your backup infrastructure. While this shouldn't be considered you only line of defence and offline / offsite backups in addition are very critical, I don't like that veeam passes on credentials to backup repositories to clients which are AD-joined.

[DChiavari]

Sorry for the delay in the reply. While indeed Veeam uses the UNC path to read/write to the shares (not sure how else you could do it with the SMB protocol?), there is no credentials exchange in clear text - everything is encrypted, plus the communication between the VBR server and other components (proxies, repositories, etc.) is always secured via TLS and authenticated via certificates.

There are, of course, wider security considerations as you mentioned: shared folders (SMB, NFS) are generally considered low-security options for repositories in the first place - while encryption is available in SMB v3 for example, immutability is not. In our Security Best Practices Guide we suggest to use different credentials and separate the management/backup domain from the main production one. An arguably simpler - albeit less secure IMO - option some customers prefer is implementing the Veeam Infrastructure in a workgroup, outside the domain.

Both credentials (and/or protocol) separation and domain segmentation provide better resilience against attacks such as credentials theft or privilege escalation.