vFRC and Veeam

[DeadEyedJacks]

Hi,
I've got a client using VMware 6.5 with vFRC, virtual flash read cache for some VMs.
Are these fully supported for Veeam Backup and Replication ?

I can see that Veeam detects the presence of vFRC cache on the drives, but we are seeing occasional VMware errors regarding allocating vFRC resources when using HotAdd backups.
Should these VMs only be backed up using Network mode? Or do we need to ensure every host with vFRC resources has a local Veeam backup proxy?

TIA

[wishr]

Hi,

It is fully supported, regardless of the chosen transport mode. Could you please let us know what kind of errors you are experiencing and how they affect daily hypervisor operations? Do you have a support case ID?

Thanks

[DeadEyedJacks]

Case# 03681283
vSphere 6.5
Four hosts in cluster
Eight VMs with vFRC enabled VMDKs
Two Veeam virtual proxies and one physical proxy / repository

Veeam Backup Error
Unable to HotAdd disk, failing over to Network mode.
coincides with
Veeam One Alert
Error message on VM on Host in Cluster:
Flash Read Cache for disk 'scsi1:0' could not be configured due to an unrecoverable error. Caching has been disabled for this device.
Initiated by: vpxuser

Does HotAdd cause VMware vFRC cache to be rebuilt on host with Veeam proxy?
Should we always be using a proxy on same host as vFRC enabled VM so it doesn't rebuild vFRC cache?
Or do we need to ensure that all hosts have sufficient vFRC free resource to duplicate the largest vFRC caches?

[wishr]

Hi,

Could you please check if increasing the vFRC space resolves the issue? I'm 99% sure it will since we attach disks to the proxy, thus vFRC resources are also required. This also explains why Network mode works.

Thanks in advance!

[DeadEyedJacks]

Unfortunately adding vFRC resource to host cluster would require client to invest in additional SSDs; so not really a troubleshooting option.

It does seem to be the case that using an off-host proxy causes errors, whereas an on-host proxy with access to the local SSDs doesn't.

Doing some further testing today.

[wishr]

I don't think it's related to the proxy mode. It may be related to the proxy version, for example, if the off-host proxy is outdated and does not support vFRC, but I presume you keep the backup components updated, so it should not be the case.

For us, it sounds like an expected behavior:
1. We are trying to attach the disk to the proxy;
2. We see that vFRC is used for this disk;
3. We try to perform the attach operation, but there are not enough vFRC resources to accomplish it;
4. We raise a valid error and failover to NBD. In the meanwhile the VM raises an alert, we detect it by VONE data collectors and raise it in the Monitor client;
5. We successfully perform a backup over NBD.

Hope it helps!

[DeadEyedJacks]

As HotAdd is the desired mode, NBD been slower, I've mitigated this by pinning a Veeam Backup Proxy to each VMware host in the cluster.
This at least ensures that the vFRC snapshot is accessed locally and hotadd used.

NB There does seem to be some underlying vSphere issue with poor tracking of free vFRC resource on each host and failure to clean-up old vFRC snapshots when VMs are vMotioned between hosts.

[wishr]

So now it works without vFRC resource increase? Also, could you please let us know how we could reproduce the aforementioned resource cleaning failure?

Thanks in advance!