MAJOR Issue with Repository in VM

[mitchellmans2203]

Hi Guys,

Any help here is appreciated as we are a lose end with VEEAM Support: 03970740

Our VEEAM Infrastructure is configured as follows:

VEEAM Backup and Replication Server:
- 4 cores
- 32GB RAM
- Sits on a 1GB Management Network in our 2ndary Datacentre

VEEAM Proxy Servers x 5 (All sit on the same ESX Host - as advised by VEEAM)
- 8 cores
- 16GB RAM
- Sits on a 1GB Management Networking in our Primary Datacenter

VEEAM Repository Server 1
- 24 cores
- 96GB RAM
- Sits on a 1GB Management Networking in our Primary Datacenter
--> 10GB Distributed Virtual Switch for ISCSI1
--> 10GB Distributed Virtual Switch for ISCSI2
*Extent E, F and G are configured with 134TB each from a Dell Compellent SC2080 SAN primarily used for Backup data --> Configured as a SCALE OUT REPO within VEEAM* --> Disks are configured with REFS 64k block size. Each extent is limited to 4 concurrent tasks, and 200mb as advised by VEEAM to try and counteract the issues below

The Proxies and Repo are replicated at the 2ndary Datacentre for backup copy jobs and backups within the 2ndary DC for stretched subnet VMs.

Unfortunately our Backup SAN went into emergency mode a few weeks back which meant the extents went offline. We have since resolved the issue but ever since... EVERY single time we run a backup (Active Full, synthetic, incremental... you name it) the VREPO server locks up and is unable to process any of the backups.

We have raised with Dell, VMWARE, VEEAM, MS - The disks never go offline if the repo isnt in use with VEEAM Backup and Replication. We have got to the point where we have deployed a new REPO - we are running some jobs to an old SAN whhich we are able to use Temporarily - however we cannot use that forever. We have no idea (nor do VEEAM) what is going on. Any help would be extremely appreciated. We have also asked VEEAM to vet the config as it was considered a 'best practice' config.

[HannesK]

Hello,
hmm, hard to guess... if I understood you correctly, then "VEEAM Repository Server 1" is a virtual machine (otherwise it could not connect to the 10GBit distributed switches). You also say, that a new repository server works fine... so I would just replace the old repository VM with a new VM (same name, same IP, just re-deploy / rescan)

The Proxies and Repo are replicated at the 2ndary Datacentre for backup copy jobs and backups within the 2ndary DC for stretched subnet VMs.

I do not understand this sentence. Is it relevant for the setup? If yes, can you describe in which situation you are using replication jobs and when you are using backup copy jobs?

Best regards,
Hannes

[mitchellmans2203]

We have ended up with re-provisioning the REPO in VMWare

All of our veeam infrastructure is on vmware all tied to the same host. According to VMware if you are connecting virtual machines all on the same host it is able to utilize 10gb networking (i dont believe this but its something vmware have told us on multiple occasions)

The point still remains we have not been able to get to the root cause of the issue and still have an issue where we believe we have over-provisioned on resource within VMWare to our proxies, and repo and probably not using all the resource for our backups. Our jobs are now completing with processing rates of around 20mb/ps