vSAN - Best Practice Local Backups (vSAN File Services?), Proxies and Remote Transport/Backups

[mriksman]

Hi,

I am trying to determine the best practice configuration for a vSAN cluster. We will have multiple sites, and due to unreliable communication links, I want everything to be performed locally at each site.

From what I have read/understand;
* I install Veeam B&R as a VM, on any one of the four vSAN ESXi nodes.
* I install a Virtual Appliance Backup Proxy on each vSAN ESXi node. This will allow the proxy that has direct access to the VMs data to retrieve the data without having to go though the network, and perform the compression.
* At this point, the data will still have to exit the ESXi that the Virtual Appliance Backup Proxy is running on, traverse the network switch and come back to the Veeam B&R VM. Right?

Where should I store the backups? Should it just be a folder on the Veeam B&R VM? Or would there be benefits using vSAN File Services?

What would be the data flow if I use vSAN File Services? I'm trying to limit network traffic as much as possible. It would be great if the Virtual Appliance Backup Proxy could perform the compression, and then without going out to the network, go directly back to the vSAN File Service share.

Would be interested to know how other people are perfroming local backups with vSAN, and no other external storage.

I presume then, there is a way to take that local backup off-site? Veeam Enterprise Manager?

[HannesK]

Hello,

I install a Virtual Appliance Backup Proxy on each vSAN ESXi node

you can, but sounds like too much management overhead for such a small environment. I would just go with one proper VM per site (start with 8 vCPU & 20GB RAM) that does everything.

traverse the network switch and come back to the Veeam B&R VM. Right?

only if configured wrong (automatic settings prevent that). The proxy writes to the repository. See user guide

Should it just be a folder on the Veeam B&R VM? Or would there be benefits using vSAN File Services?

local disk formatted with REFS (skipping the other questions because nobody should want to use NAS shares in 2022 as backup target )

I presume then, there is a way to take that local backup off-site

yes, backup copy jobs or capacity tier in copy mode

Note: I assume that each location has it's own VCenter. If not, then your are not winning too much, because if VCenter connection drops, then the backup software cannot create snapshots etc.

Best regards,
Hannes

[mriksman]

Thanks for the reply.

Wow. I was way off...!

you can, but sounds like too much management overhead for such a small environment. I would just go with one proper VM per site (start with 8 vCPU & 20GB RAM) that does everything.

Veeam KB 2273 here and the blog post here states

If there’s a local proxy to the data, Veeam gives priority to that proxy to reduce network traffic. This is because the data object may not necessarily reside on the compute host since with vSAN there is no data locality. To accomplish this, it is recommended that you have a virtual proxy, per vSAN node.

Which to me indicates that a Virtual Appliance must be configured on each ESXi vSAN node to prevent unneccessary network traffic.

traverse the network switch and come back to the Veeam B&R VM. Right?

only if configured wrong (automatic settings prevent that). The proxy writes to the repository. See user guide

local disk formatted with REFS

If the Veeam B&R VM is the repository with a 'local disk' formatted REFS, then the proxy will write to this repository. So if a VM's components reside on the vSAN cluster on another ESXi host, then it will have to exit the ESXi host where the VM data resides, pass the network switch, and go to the Veeam B&R VM that has the ReFS drive, won't it? There is a good little article with a picture explaining this here.

(skipping the other questions because nobody should want to use NAS shares in 2022 as backup target )

How come this isn't mentioned in their blog?

Note: I assume that each location has it's own VCenter. If not, then your are not winning too much, because if VCenter connection drops, then the backup software cannot create snapshots etc.

Crap. I did see this on the blog post.

Since vSAN is distributed on local storage across multiple hosts, Veeam obtains information about the data distribution from vCenter.

If that is the case, then there is no point in running Veeam B&R locally at each site, as I'll only have one instance of vCenter located at the main site for management.

So how can Veeam B&R operate locally without communication with vCenter? Or is it not possible? And installing vCenter at each location would likely need additional vCenter licenses and the vCenter Enhanced Link Mode can only centrally manage 15 remote vCenter instances.


So I circle back to my question. What is the best practice to create local backups on multiple small vSAN cluster (four nodes)?

[mriksman]

It sounds like you have suggested

• Main site has vCenter and Veeam B&R that starts/manages remote backup jobs (via backup proxy) and 'backup copy jobs' from the remote site to an off-site location
• Each remote site has a single Windows Server VM that serves as both the Backup Proxy and the local ReFS repository
• Accept that data moving from each of the vSAN ESXi node storage will pass through the 1Gbps network switch (except if the data is on the same node as the Backup Proxy/Repository)
• No solution to long communication outages between the main vCenter/Veeam B&R and the remote sites. Jobs will no longer run. The only solution is local Veeam B&R (no additional licenses required) and local vCenter (additional licenses required).

Is that correct?

[HannesK]

Hello,
from the KB article

You may however want to reduce the network traffic

"may" means "it's possible, but not required". There was a recommendation some years ago (time of the blog post for example), when we recommended one proxy per host. We stopped that recommendation because reality showed little advantages with the growth of network bandwidth. If you see that bandwidth is too low, you can still add proxies

My suggestion was to put everything inside one machine for such small environments to keep it simple. Sure once can make it complex, but I like the KISS principle

The VSAN files blog post is about backing up VSAN file services. I was talking about that NAS shares are a bad Veeam backup repository in general and recommended REFS (well, also XFS would be fine, but that would require Linux - if you like Linux, also fine)

As VCenter is the instance managing VMware, it's the only place we can communicate with to find machines, create snapshots etc. Yes, a design that can deal with network outages would require a VCenter in each location.

Your understanding is correct. That's what I would recommend.

Best regards,
Hannes

[mriksman]

Great reply. The fog is clearing a little

Which solution is better?

• Veeam B&R at each location that has the ReFS repository and performs the backup proxy. Veeam Backup Enterprise for central management of all remote Veeam B&Rs. Or,
• Veeam B&R at the main site. Each remote site has a Windows Server VM with an ReFS local drive and performs the role of the backup proxy.

Thanks.

[HannesK]

"better" is more a personal preference in that case, I think. I would go for one central VBR server, because then everything including updates can be managed centrally

[mriksman]

@HannesK
Thanks for your reply.

I don't suppose you have any pros/cons? Why would someone have remote Veeam B&Rs managed by Veeam Backup Enterprise Manager as opposed to a single Veeam B&R instance that manages all the proxies/repositories?

Perhaps missing tools/integration with vCenter? Something else I'm missing? I think maybe if the remote sites themselves have 100s of VMs, and there are many remote sites, a single Veeam B&R might struggle with SQL Express as a database; so separating them might be better in that regard. For me, I have 7 small remote sites and a total of 80VMs spread out between them. So perhaps for my small deployment, a single Veeam B&R makes more sense? There will also be a handful of physical endpoints to backup. Does that make a difference? I presume the local backup proxy and repositoy does all the work, and the remote Veeam B&R server manages the job (so no large data transfers between Veeam B&R and the remote site?).

Thanks.

[HannesK]

Hello,

So perhaps for my small deployment, a single Veeam B&R makes more sense?

that's what I recommended above from the information given I repeat my recommendation after reading about the new details provided. One VBR instance can handle many thousand machines... no need to worry about a few hundred machines.

Best regards,
Hannes

[RobTurk]

Why would someone have remote Veeam B&Rs managed by Veeam Backup Enterprise Manager as opposed to a single Veeam B&R instance that manages all the proxies/repositories?

Here's an example: You mentioned unreliable links in your first post. Having a VBR server at each remote site allows backups and restores to run at those sites even when a link to the central location is down.
The downside is that you may need more resources at each remote site.

Best regards,
Rob

[mriksman]

@RobTurk
With vSAN, doesn't VBR need to be able to communicate with vCenter to perform backups and restores?

[HannesK]

Hello,
VBR has to be able to talk to VCenter. VCenter is the command and control center that for example triggers snapshots.

The port requirements are the same for VSAN and other storage.

I guess my colleague has a setup in mind, with one VCenter per location.

Some types of restore work without VCenter. For example file level restore with "copy to". Things like VM restore require VCenter (I don't know, whether adding an ESXi host of a VSAN cluster directly could work. but that's nothing I would consider as recommended design)

Best regards,
Hannes