EMC Scale I/O

[chrisBrindley]

I was wondering if Veeam has tested or anyone has tested backups against SCALE I/O storage, I not saying our backup storage is SCALE I/O but our vm storage is, we went entirely hyper-converged with VSAN and had no issues, in the last month we decided SCALE I/O was a better product.
Since we switched we are seeing very strange behaviour on backups, we will see a server with 4 disks on the same LUN and 2 of the disk won't hot add, then the proxy will then throw an error Reconfigure virtual machine
nxuatsrvbkp07 Invalid configuration for device '0'. These issues are constant. I also see the proxy trying to mount scale IO luns that aren't part of the cluster and then fails to hot add.

I have also decided to try and force the proxy to only use the specific datastore it can hot add but it just errors that direct SAN is not available failing over to network mode.

Here is the interesting part we have a management cluster on the same V-sphere Veeam environment as well as a VSAN cluster, the management cluster uses an NFS mount for its storage, we see no issues when backing up any servers on these clusters.

I have a case opened but wanted to ask the brains here if SCALE I/O is even supported?
Case # 02160601

[dellock6]

For what I know, ScaleIO uses a server client architecture, so it seems hard to me that our proxies can do Direct SAN backups like they would do for a regular iscsi or FC block storage. How are the proxies configured? In an hyper-converged solution, I would go for hotadd anyway. Also, when you say the proxy tries to mount LUNs outside of the cluster, what do you mean? The hotadd against other VMs? In this case, indeed a proxy can only hotadd VMs in the same cluster, so you would need to have at least one proxy for each cluster.

Luca

[chrisBrindley]

I have been looking through our environment all day, our proxies are base setup will hot add if possible and fail to network if not.
Following the job through the steps, the snapshot is taken the proxy mounts the backup server drive, i am not sure i follow what happens next but after a short pause the proxy server will dismount the drive, this is successful but throws the error
Reconfigure virtual machine
nxuatsrvbkp02
Invalid configuration for
device '0'.

The job then states the following
using backup proxy bkp02 for hard disk1 (hot add)
unable to hot add source disk, failing to network mode

This is happening on over 750 servers

I have now reboot the V sphere appliance all proxy servers and Veeam Manager / SQL server. Rebuilt an entire new veeam environment attached to the same V sphere instance, same issues.
case opened with VMware will post my finding in case this helps someone in the future

[dellock6]

There are many limitations about hotadd and scenarios where it may not work:
https://www.veeam.com/kb1054

I would suggest to also check this list, and maybe also open a ticket with Veeam support.

[chrisBrindley]

Posting the follow ups to this issue as I Believe the Veeam engineers need to look at this issue as SDS is the way forward for a lot of companies.
First, off thanks to Fedor Belov from Veeam, he was going around my environment like a crazy person trying to figure out this issue.

Issue 1. Proxy servers were built with one drive and using Paravirtual SCSI Controller, these proxy servers would hot add the drives but couldn't read them and eventually failed.We had to switch to LSI Logic SAS controller and this solved the issue.
Issue 2. Multiple Scale IO luns mapped to Clusters, the proxy servers seemed to get confused with SDS, it was picking proxy servers that could access the storage but they were in different clusters, they would not hot add the drives and throw the invalid configuration for Device 0. We assigned one Lun to each cluster and everything seems good now

It's worth noting we also run cloud link encryption so not sure if this somehow confuses the proxy servers.

My hope to Veeam is installing V-sphere 6 and setup Scale IO from EMC and verify this is supported