Possibility of combining proxies and repositories on the same host when working with vSAN

[disp.tac@rubytech.ru]

Hello,

Case ID: 05364224
Product: VBR
Components: Backup
Description:
When a VM's disks reside on a vSAN datastore and the VM is used as a proxy and as a repo together, HotAdd mode is not available and it is only possible to use NBD:
[14.03.2022 13:23:35] Info [ProxyDetector] Processed VM has disks on vSan, but some disks of proxy VM are not on vSan, so HotAdd is impossible and can't failover to network

We want Veeam to add the possibility of combining proxies and repositories on the same host when working with vSAN.

Thanks.

[HannesK]

Hello,
and welcome to the forums.

Sounds like failover to NBD is disabled in the transport mode settings (or I misunderstood the request)

Best regards,
Hannes

[disp.tac@rubytech.ru]

Hannes,

It is not possible to activate failover on nbd - the traffic will go through the firewall, which will create a parasitic load on it and can lead to negative consequences. Also we already opened cases at technical support (case ID's: 05330389 and 05364224).

[foggy]

Consider using a separate repository, using the proxy itself prevents you from the ability to use hotadd for processing in the case of vSAN.

[disp.tac@rubytech.ru]

Well, that's why we created that topic. We want to add the possibility of combining proxies and repositories on the same host when working with vSAN. When we asked open Feature request at technicall support, they suggested us to open topic here, to be able follow the status.

[foggy]

Ah, ok, this is a feature request. Understood and noted, thanks!

[HannesK]

as the request sounds like a (risky and expensive) corner case to me... what is the overall goal of the design? having virtual repositories on production storage (I guess VSAN is production storage as you try hot-add) sounds expensive to me. It also imposes risks of chicken egg problems (how to restore if VSAN goes down?). From a security perspective, it also is a risk that an attacker who gains access to VCenter can delete the repository.

[disp.tac@rubytech.ru]

Hannes,
the purpose of this configuration is to localize backup traffic inside the proxy server, the virtual machine (proxy + repo) reads productive data with hotadd and adds it to his local repository, which is served to it by SAN.
If VSAN fails, the entire virtualization farm will fail, because architecturally there is only VSAN (with the exception of disks for the backup repository). There will be nowhere to restore data.
>From a security perspective, it also is a risk that an attacker who gains access to VCenter can delete the repository.
Exactly like any proxy / repository, regardless of its connection. Protection against this is considered at other levels of storage of additional backups.

[soncscy]

Do I get that the reason you need this setup is because you're kind of stuck with it? If so I get it, but the above ideas still apply. The threat model isn't just about "can we perfectly protect against the attacks" but also mitigating the risk by creating additional separated objects they have to overtake for a successful attack.

But, if you're stuck with it you're stuck with it; I suppose containerization would help here; run a linux proxy, set up container (LXC or pods or something) to host the repository, and then you just go through the container stack for the networking and the traffic is "local". It feels a bit dirty, but I suppose it gives you Proxy + Repo on vSan _today_. I've not really read about how the performance usually goes, but even if it's fairly slow, after the initial full, you should be getting pretty okay performance for the incremental runs, no?

But I'd hate to troubleshoot it...

[Dream_On]

I think, the problem here in your firewall design.

[disp.tac@rubytech.ru]

Do I get that the reason you need this setup is because you're kind of stuck with it? If so I get it, but the above ideas still apply. The threat model isn't just about "can we perfectly protect against the attacks" but also mitigating the risk by creating additional separated objects they have to overtake for a successful attack.

But, if you're stuck with it you're stuck with it; I suppose containerization would help here; run a linux proxy, set up container (LXC or pods or something) to host the repository, and then you just go through the container stack for the networking and the traffic is "local". It feels a bit dirty, but I suppose it gives you Proxy + Repo on vSan _today_. I've not really read about how the performance usually goes, but even if it's fairly slow, after the initial full, you should be getting pretty okay performance for the incremental runs, no?

But I'd hate to troubleshoot it...

This is good workaround, but we would like to have the possibility of such a merge "out of the box". As for now, we dont see any reasons for developers not allowing that kind of design.

[micoolpaul]

Hannes,
the purpose of this configuration is to localize backup traffic inside the proxy server, the virtual machine (proxy + repo) reads productive data with hotadd and adds it to his local repository, which is served to it by SAN.
If VSAN fails, the entire virtualization farm will fail, because architecturally there is only VSAN (with the exception of disks for the backup repository). There will be nowhere to restore data.
>From a security perspective, it also is a risk that an attacker who gains access to VCenter can delete the repository.
Exactly like any proxy / repository, regardless of its connection. Protection against this is considered at other levels of storage of additional backups.

To focus on the second point first: if a repository is physical, nope, compromise of vCenter wouldn’t result in the deletion of the repository.

As for the first point, Veeam can confirm but it depends if VMware’s VADP supports this scenario.

Finally your final sentence sounds like you’re using other repositories for backups off of vSAN, I’d only consider backups not on your production hardware to be actual backups, it’s like counting RAID as a backup, the number of scenarios whereby you’re saved by these backups is smaller than a standalone repo.

I can only assume to these other backup repository’s you’re using backup copy jobs, so surely this is creating firewall load too? Or are you relying on having a repos in a particular subnet to avoid the firewall?

Also vSAN is going to require multiple nodes. What happens when it needs to process VMs on another host? How can you prevent failover to network mode?

[HannesK]

Hello,

localize backup traffic inside the proxy server

That is possible today, by running a proxy on the same ESXi host in the same VLAN. VM affinity rules can help with that. By running both VMs on the same host, the network traffic never hits the physical network infrastructure (which seems undersized here). If network bandwidth is the issue, then I only see proxy & repository outside the VSAN cluster. Backup mode: NBD. In all other scenarios, there is network overhead for reading and writing on VSAN.

There will be nowhere to restore data.

right... even if the issue finally gets fixed, the backups are also lost. I suggest again to implement the 3-2-1 rule

Best regards,
Hannes