VEEAM B&R poor performance writing to FC storage target

[chrisb]

Hello,
I am experiencing poor performance when backing up VMs using VEEAM 4.1.2 to fibre channel SAN storage - I have a 4Gbps backup SAN with an HP MSA2312fc storage array and tests with IO Meter show read and write speeds of 145MB/s and 239MB/s respectively, but VEEAM backups run at a maximum of 30MB/s, with some jobs slowing down to as low as 3MB/s at times.

The result is that a job with 19 VMs (around 834GB in total) take almost 16 hours to complete.

The job/s in question are configured to us VMware vStorage API, in SAN with failover mode - VMware tools quiescence and CBT is disabled, and compression is set to optimal.

During testing, I configured a job to write to local storage on the backup proxy and speeds increased to 130MB/s, which is the sort of speed that I was expecting to see - The problem is that, as soon as I point the job to SAN storage (which I know can perform at this speed), VEEAM performance drops drastically.

Are there any known issues with VEEAM writing to FC SAN storage?

Many thanks

Chris

[Vitaliy S.]

Hello Chris,

Could you please specify how did you present SAN LUN to the backup server? Have you been using HBA driver to mount an NTFS LUN to the Windows machine with the backup console?

Besides, could you please elaborate why you've decided to disable CBT for your backup jobs? Using CBT would bring you significant performance improvements assuming that your VMs meet these requirements.

Thank you!

[chrisb]

Hi
The storage is presented as a Windows volume which is formated as NTFS and configure as drive D:\ on the server - The connection is via an HP fibre channel HBA.

The job in question has CBT disabled because up until recently the VMs were not v7 hardware. I have some jobs with CBT enabled but this made no difference to performance - The jobs still run very slowly with CBT enabled.

Many thanks

Chris

[Vitaliy S.]

Chis, thanks for the clarifications. So as far as I got it right, you've just chosen D drive as a destination target in the backup console. And as a further step of troubleshooting could you please upload the backup files that you have on a local storage to the D drive using Windows Explorer. What speed rates do you get? Besides, what is the CPU load when backup jobs are running?

By the way, If you have a poor performance even with the CBT enabled, I would suggest contacting our support team providing all the logs just to be sure that CBT does actually work for your VMs. Thanks!

[Gostev]

Try uploading and downloading files from target LUN to Veeam Backup server at the same time. I've seen one occasion when mixed read/write IO was causing misbehaving HBA to go crazy. Upload - extremely fast, download - extremely fast, mixed - a few MB per second only.

[chrisb]

Thank you Vitaliy and Gostev - On further investigation, it seems that the issue might be related to storport.sys causing performance degradation in read/write operations.

I have updated the storport driver and file copies are improved so hopefully tonight's backups will be ok.