Backup Design with 10Gb Network and Fibre Channel Storage

[ITaaP]

I need to setup Veeam in a VMware infrastructure with 10Gb networking and 8Gb FC. I've never worked with FC before and wanted to get an idea of different setups. The backup repository will be on the 10Gb network. There is no direct storage integration. I figured network mode is probably the best solution to use for this deployment. Currently there is no separate storage network, well IP storage network, since it is all FC. Being that all traffic is going over the same switches, I am wondering if it is necessary to create a separate VLAN and subnet to be used for backups?

[csinetops]

I'll just throw in my .02 cents because my setup is a lot like you are describing. My storage is all 8GB FC and network is 10GB between hosts. My backup proxies are virtual with large .vmdk's for repositories, some on local DAS and some on older SAN. Your backup ESXi host will need to be able to see your production Datastores. Then you can just leave backup mode on auto and Veeam will use Hot Add mode or fail over to network if need be. Really no reason to push that amount of data over the network when the hypervisor can hot add the VM disks to your proxies for backup.

If it for some reason does have to fail over to network mode I haven't noticed any difference in speed, just more load on the switching infrastructure.

I've been setup this way for almost 3 years now and it works good. Let me know if you have any other questions.

[vClintWyckoff]

Just curious why not use Direct SAN access since you're all FC for storage? HotAdd places additional load on your hosts plus the additional processing time to perform the HotAdd. DirectSAN would read the blocks straight off the array so no overhead of the ESXi environment.

[ITaaP]

I'll just throw in my .02 cents because my setup is a lot like you are describing. My storage is all 8GB FC and network is 10GB between hosts. My backup proxies are virtual with large .vmdk's for repositories, some on local DAS and some on older SAN. Your backup ESXi host will need to be able to see your production Datastores. Then you can just leave backup mode on auto and Veeam will use Hot Add mode or fail over to network if need be. Really no reason to push that amount of data over the network when the hypervisor can hot add the VM disks to your proxies for backup.

If it for some reason does have to fail over to network mode I haven't noticed any difference in speed, just more load on the switching infrastructure.

I've been setup this way for almost 3 years now and it works good. Let me know if you have any other questions.

My backup repository is a dedicated 40 TB storage server that is only going to be on the 10Gb network. So even using the Virtual Appliance Mode (hot-add), the data still goes over the 10Gb network. Plus I really don't care for hot-add mode as there is a better chance of VM stun happening and other issues I have come across. If it was a 1Gb network, then yes hot-add would be the way to go. With 10Gb, Network Mode is a much better option, but I appreciate the recommendation.

[csinetops]

Just curious why not use Direct SAN access since you're all FC for storage? HotAdd places additional load on your hosts plus the additional processing time to perform the HotAdd. DirectSAN would read the blocks straight off the array so no overhead of the ESXi environment.

I would love to use storage integration with our Netapp(s) but we have VM's with disks that span different datastores. From what I have read this isn't supported.

[ITaaP]

Just curious why not use Direct SAN access since you're all FC for storage? HotAdd places additional load on your hosts plus the additional processing time to perform the HotAdd. DirectSAN would read the blocks straight off the array so no overhead of the ESXi environment.

Simple, the backup repository goes not have FC available.

[csinetops]

My backup repository is a dedicated 40 TB storage server that is only going to be on the 10Gb network. So even using the Virtual Appliance Mode (hot-add), the data still goes over the 10Gb network. Plus I really don't care for hot-add mode as there is a better chance of VM stun happening and other issues I have come across. If it was a 1Gb network, then yes hot-add would be the way to go. With 10Gb, Network Mode is a much better option, but I appreciate the recommendation.

The setup you describe will work then. That's interesting you have stun issues, I have never experienced that. It takes a snapshot regardless if it is using hot-add or network mode.

[vClintWyckoff]

Right, I'm not talking Storage Integration, ie - backup from Storage Snapshots...I'm talking directly zoning in the VMFS LUNs to your Proxy Servers with the FC HBAs in them. Which would then send the read data over 10Gb network to the Repository Server.

[ITaaP]

That is a possibility, but like I said, I have never used FC before. Could you please provide a little bit more detail on how this setup works?

[vClintWyckoff]

Could you please provide a little bit more detail on how this setup works?

Certainly, but the best place is to head out to the Best Practices Guide @ http://veeam.expert/bp

This has the answers to all those questions and more actually

[ITaaP]

I was actually already using that guide. Well, the more recent one for v9 update 2. Looking at http://bp.veeam.expert/resource_plannin ... t_san.html still leaves a lot of questions for me since I am not familiar with FC, zoning, etc. I studied that to pass my VCAP exam, and that is about it. I have zero practical experience. The other issue is this is for a customer and I have very limited insight into their environment. While DirectSAN provides the best performance, I am curious if it is really the best option with my limited knowledge of deployment. Looking at Rubrik, they would also utilize the API and backup over the network.

[dellock6]

For the setup, the explanation is pretty easy to undestand, your Veeam physical repository needs to be configured exactly like an additional ESXi host in terms of zoning.
For the benefit, the biggest advantage is that backups are executed over the FC fabric, with ZERO traffic going via the esxi network. This can be a huge benefit as data goes straight from storage to repository, without traversing the ESXi stack. As much as you can tune any possible component in between, that's an additional layer that data has to cross.

[ITaaP]

Probably pretty easy for someone that has used FC. When I edit the settings on a VM in my own environment, I don't see anything about a FC device. I am guessing to set this up it would require the FC HBA configured for passthrough and then add a PCI device to the VM?

[dellock6]

Wait, the best approach is to use physical Veeam proxies to leverage DirectSAN over FC fabric. If your proxies are going to be VMs, than better to use hotadd or network mode...

[ITaaP]

Then Network Mode it is. Curious though, is it fairly common to use physical servers as the proxy? I would imagine to backup a few hundred VMs every night, it would require a decent amount of CPU.

[dellock6]

Yes, it becomes common among medium and large customers, because in order to process many vm concurrently you need many cores; you get those cores from many virtual proxies, but then you are consuming resources from the same environment you are protecting ; or you can use dedicated physical proxies, where the processing power is guaranteed as it's not shared with other processes. This is valid for both direct san but also network mode.

[tomhkr]

Our storage is also 8 Gb FibreChannel and our Veeam backup and proxy (same machine) server is a physical 16 GB / 12 core Xeon server. It has never hit any notable amounts of CPU usage doing its nightly 110 VM jobs.

The backup and proxy server has a dual 8 Gb FibreChannel adapter providing DirectSAN access to the storage. It's pushing about 500 Mbyte/s, the limit in my case is a deduplicating storage unit coupled to the proxy with 10 GbE.

I warmly recommend this setup, it has worked very well and cut down our backup times compared to our previous storage that could deliver about 150 Mbyte/s at peak.