How to improve source load: 99%

[nufael]

Hi,

I've read some of the topic posted in this forum but I can't really found the exact answer that can help me to improve the source load during backup is running.

My environment:

- Veeam Backup and Replication version: 6.5

- Vcenter:5.5 (VMs datastore connect to EMC VNX 5500 unified storage solution using Fiber Channel)

- Veeam Backup (admin) and proxy server located in the same physical server

- Repository: Resides in EMC VNX 5500 unified storage solution (Using CIFS)

- Transport mode: Auto Selection

- Backup mode: Forward Incremental

I've run the full backup job for one VM with size = 60G (15.5 used). Here are the summary:

Processing rate = 22MB/s
Bottleneck = source

Load: Source 99% > Proxy 76% > Network > 16% > Target 31%

Based on one article stated that source is the source (production) storage disk reader component. The percent busy number for this component indicates percent of time that the source disk reader spent reading the data from the storage. By referring to my backup job summary as stated above, it seem that the source storage disk reader busy while reading the data. If refer to other article, most of it emphasize on how to optimize the proxy load by provisions another proxy server (virtual mode).

May I know what is the mechanism in order for me to improve the source load performance and at the same to optimize the vcenter load during the backup job

Thanks and Regards

[Vitaliy S.]

Hi Nufael,

If you storage cannot provide more performance and backup job cannot read data any faster, the only way to improve this is to upgrade the hardware. However, there is a couple of things that I would like to check in your setup:

1. Can you please confirm you're using Direct SAN backup mode? Can be done via job sessions, just search for [san] tag there.
2. Is there any special reason you're using Veeam B&R v6.5 that does not have full support for vSphere 5.5?
3. Was it a full or incremental backup job run?

Thank you!

[nufael]

Hi Vitaly,

Appreciate your response

To answer your question.

1. Automatic mode with enable failover to network mode. If Im not mistaken the SAN access mode is not supported with NFS storage (cifs repository)

2. No specific reason and I believe I'm not aware on the weaknees on the integration between Vsphere 5.5 and Veeam Backup and Replication v6.5. Can you advice me on the suitable version that can suit with vsphere 5.5?

3. Yes, full forward incremental backup job

Thanks and Regards

[Vitaliy S.]

Nufael,

1. I was not talking about target, but rather VM source datastore. Please take a look a this topic for more details > Direct SAN Access Mode. Try to configure direct SAN mode in order to have better job performance.
2. Here you go > [ALL VERSIONS] Current version is 7.0 Patch #4 (7.0.0.871)
3. Actually I was asking what job session was that. Was it a full/initial job run?

Thanks!

[nufael]

1. By referring to the statistic and report via the job session, there is no [san] tag. If my step is incorrect please help to lead in the correct way.

Referring to your advice, in order to get better job performance, the best way is to configure direct SAN mode. My environment is, no FC HBA and iSCSI target available at a moment. I'm using CIFS for the target location (EMC Unified Storage). Any best advice?

2. Thanks, appreciated

3. Full

Thanks

[foggy]

1. By referring to the statistic and report via the job session, there is no [san] tag.

That means that network backup mode is used. Do you see the VMFS volumes in the Disk Management snap-in on the Veeam B&R server?

Referring to your advice, in order to get better job performance, the best way is to configure direct SAN mode. My environment is, no FC HBA and iSCSI target available at a moment. I'm using CIFS for the target location (EMC Unified Storage). Any best advice?

Vitaliy's advice regarding direct SAN mode applies to reading data from the production datastore, not writing them to the backup target.

[nufael]

Hi Foggy,

1. No. Do you think I need to have FC HBA installed at Veeam Backup server or prepare iSCSI target in order to boost the backup performance? Would it also improve on source load?

2. Meaning that I need to check at the Vsphere setting on how they reading data from datastore which located at EMC unified storage?

regards

[foggy]

Basically, everything you need to enable direct SAN mode is described in the FAQ topic referred to above:

Q: How to configure the new backup proxy server for direct SAN access?
A: Disable automount (done automatically when backup proxy server components are installed), install and configure SAN HBA card (or iSCSI Software Initiator), give the backup proxy server machine access and read permission on VMFS LUNs (if applicable to your SAN), and ensure that you are seeing your VMFS volumes in the Windows Disk Management snap-in. Refer to the following thread for more information specific to certain SAN makes and models: Configuring new VEEAM Backup Server for SAN.

[Vitaliy S.]

Does you production storage have FC or iSCSI connection? If one of these, then you can configure Direct SAN mode and read data more efficiently. If there is no opportunity to do that, then I would suggest switching to HotAdd backup mode. This will require virtual proxy servers on the hosts/clusters you're backing up VMs from.

[nufael]

Production storage for Veeam backup repository or production storage for VM datastore? For VM datastore we have FC connection to ESXi host but for VBR repository we dont have iFC or iSCSI connection.

Another question, referring to your advice earlier to upgrade VBR v6.5 to 7 due to fully support for vSphere 5.5. Is there any major impact if I'm still using v6.5 with vSphere 5.5 rather than v7?

thanks

[nufael]

I think i understand your statement wrongly. This should clear the cloud. In my proxy environment, there is no FC or iSCSI connection to production storage. FC and iSCSI connection are used to communicate ESXi host with storage datastore (resides in EMC) only. Proxy server only used ethernet connection to retrieve data from source.

[foggy]

If there's no ability to use direct SAN mode, then consider deploying the virtual proxy to be able to use hotadd for data retrieval.

[Vitaliy S.]

Another question, referring to your advice earlier to upgrade VBR v6.5 to 7 due to fully support for vSphere 5.5. Is there any major impact if I'm still using v6.5 with vSphere 5.5 rather than v7

The major impact is the following - if you run into any issues with this configuration, our support team will not be able to help you, as VBR v6.5 does not officially support vSphere 5.5. Thanks!

[foggy]

You can also find some usefull information in this thread.

[Vitaliy S.]

FC and iSCSI connection are used to communicate ESXi host with storage datastore (resides in EMC) only. Proxy server only used ethernet connection to retrieve data from source.

Just to make sure we are on the same page, you have production datastores that are connected via iSCSI and some via FC? If you don't have any spare HBA adapters for FC storage, then proceed with hotadd backup mode. As to the iSCSI connected datastores, then use Windows iSCSI initiator to present LUNs to the proxy server and then use direct SAN mode to retrieve VM data from the storage.

Here is a link on how to configure iSCSI connection from the proxy to the storage > http://www.jpaul.me/?p=334

[nufael]

Ahhh, sorry my bad. My production datastore are connected to ESXi host via FC ONLY.

My proxy server is a physical server (not a VM server thus no connection to production datastore via FC).

Correct me if I'm wrong, I can't use Direct SAN backup mode since FC or iSCSI connection are not available in between proxy and production datastore.

Thanks

[dellock6]

Correct, in order to use DirectSAN the proxy needs to be connected into the same storage fabric as the ESXi servers, regardless it's fc or iscsi.
you have two options:
- install an FC hba into the physical proxy and use DirectSAN
- add a virtual proxy running inside the same vSphere cluster and use hotadd mode

Oherwise you are only left with network mode.

Luca.

[nufael]

In my case, I've multiple ESXi host. Do I need to add multiple virtual proxy to every of each host?

Another thing to note is that, All VMs in ESXi host are connected to datastore using FC connection. If I add virtual proxy it will obviously running in ESXi host environment and use FC connection to production datastore, can it also consider use Direct SAN access mode?

Thanks

[dellock6]

No, if one ESXi can see the shared FC storage, only one proxy is enough. You need a local proxy only if you have local datastore and you still want to use HotAdd.

No, the FC connection is in regards of ESXi and FC Storage Array. Virtual proxy will use HotAdd.