Job Performance.

[JosueM]

Good day everyone,

I'm getting around 30MB/s processing rate for local replication.

Backup and Proxy is a VM (6 processos and 12 GB RAM) with access to both disk cabins, production is a dell MD3200i with 12 disks SAS 15K Rpm and LUN is raid 5, destination SAN is a DELL MD3220i in the same LAN with 12 disks SAS 10K Rpms Lun in raid 5.

2 Esxi Servers in a cluster , DELL R710 with 2 Intel Xeon X5680 @ 3.33Ghz and 144 Gb RAM.

Both ISCSI SAN at 1Gbps each accesed by esxi host with multipath/roundrobing (4 dedicated nics on each server for this traffic)

Veeam version is 7.0.0.690. Wonder if that performance is "normal" for such config?

Thanks in advance.

[Vitaliy S.]

Hello Josue,

What does bottleneck statistics show for this job?

Thank you!

[JosueM]

Hello Vitaliy,

Some jobs bottleneck is target and in others is source.

The veeam backup and proxy vm seems to have low usage of CPU and Memory.

[Vitaliy S.]

Bottleneck stats should be the same for all VMs assuming that you are replicating them from the same source hosts with the same backup mode selected. Can you please post full bottleneck stats for the replicated VMs? Also is 30 MB/s reported for the full run or incremental job pass?

[JosueM]

I noticed that first replication the bottleneck was target on subsequents it is source. Also noticed that for some reason not using hotadd and is failing over to network.

[Vitaliy S.]

In this case the performance you see is somewhat expected. Could the failover be caused by one of these hotadd limitations? Does your proxy VM have access to all datastores you're backing up VMs from?

[JosueM]

The VM has added small disks for each SAN at hardware level, but noticed those disk are not initialized and appears as offline at guest (windows) level. Will initialize them and try again.

[Vitaliy S.]

You do not need to initialize them, disks should just be visible in the VM (proxy) configuration. Once VMs disks are mounted, our proxy server will start backing them up.

[JosueM]

Did that and see that some disk are mount on the proxy vm at windows level storage management but the job still using network instead of hotadd. Will search thru the link above and hope that performance is related to the hotadd feature.

[JosueM]

In this case the performance you see is somewhat expected. Could the failover be caused by one of these hotadd limitations? Does your proxy VM have access to all datastores you're backing up VMs from?

Vitaliy from the link above only item I could think a possible cause is the VM has LSI Logic SAS adapter, wich is the default adapter when creating a new VM on vmware for this operating system.

Do you think it should be changed on the veeam VM ?

[Vitaliy S.]

Hmm...job log would have more information on the reason of the failover. Can you please contact our support team to review your backup job log? BTW, do all other VMs failover to the network mode as well?

[JosueM]

Hello Vitaliy,

Already opened a case on support.

Other replications jobs are offsite and they use hotadd properly, some VMs of these jobs are stored on the same SAN as the one that is failing over to network. For these other jobs main bottleneck is target.

[Vitaliy S.]

I'm getting around 30MB/s processing rate for local replication.

Other replications jobs are offsite and they use hotadd properly

Now I'm a bit confused...if you're replicating VMs offsite, then it is recommended to have a second Veeam proxy server in the offsite location. Do you have it configured?

[JosueM]

I just commented about others job since you asked if there were other jobs also failing over to network. For this case the only job that failover to network is the local replication , the offsite are using hotadd and yes there is a remote proxy for them.

That's why I suspect the local replication low processing rate could be related to the job failing over network instead of hotadd. Weird is I did open a console to the proxy during the job is running and see the source VM disks are mounted on the proxy but for some reason does not use hotadd.

[Vitaliy S.]

Ok, let's wait for our support team response after reviewing your job log files.

[JosueM]

This is weird after 5 days of posting the ticket, the support guy still trying to understand the issue!

I hope they can redirect me with another person, this is not the perception I had about veeam support.

[Vitaliy S.]

Josue, if you're not satisfied with the level of support provided, you can always ask to escalate your ticket to another engineer.

[Gostev]

Or you could click "Talk to a Manager" button on the support portal, and this will go straight to the support management DL.

However, please also appreciate that most likely, they are researching an infrastructure issue here for you (and not the issue with the code), as we know for the fact that the actual backup engine can dish out hundreds of MB per second if infrastructure allows. From my experience, most other vendors would simply ask you to hire a consultant to troubleshoot this. However, we always try to find and fix an issue for you, since it involves using our product, and we want you to be successful with your implementation.

Additionally, performance issues are not exactly the highest severity ones, and the SLAs for your case is set as for a low severity case. During "hot" times such as after a new major release, this may directly translate into slower responses from support.

Finally, I'd like to remind that you must include support case ID when posting about technical issues on these forums, as requested when you click New Topic. I see you still did not post it, after 7 days. Without the support case ID, we cannot even check on the current status of your case, or facilitate resolution anyhow. More importantly, you want to include support case ID because moderators will periodically prune all topics about technical issues without case ID included (this is also explained when you click New Topic).

[JosueM]

I forgot to post it before. Case # 00446441

[JosueM]

Finally support tech did a remote session to verify. The solution was to force proxy to work in network mode since for an unknown reason the hotadd is not working properly.