Discussions specific to the VMware vSphere hypervisor
Post Reply
nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

How to improve source load: 99%

Post by nufael » Jun 18, 2014 8:32 am

Hi,

I've read some of the topic posted in this forum but I can't really found the exact answer that can help me to improve the source load during backup is running.

My environment:

- Veeam Backup and Replication version: 6.5

- Vcenter:5.5 (VMs datastore connect to EMC VNX 5500 unified storage solution using Fiber Channel)

- Veeam Backup (admin) and proxy server located in the same physical server

- Repository: Resides in EMC VNX 5500 unified storage solution (Using CIFS)

- Transport mode: Auto Selection

- Backup mode: Forward Incremental

I've run the full backup job for one VM with size = 60G (15.5 used). Here are the summary:

Processing rate = 22MB/s
Bottleneck = source

Load: Source 99% > Proxy 76% > Network > 16% > Target 31%

Based on one article stated that source is the source (production) storage disk reader component. The percent busy number for this component indicates percent of time that the source disk reader spent reading the data from the storage. By referring to my backup job summary as stated above, it seem that the source storage disk reader busy while reading the data. If refer to other article, most of it emphasize on how to optimize the proxy load by provisions another proxy server (virtual mode).

May I know what is the mechanism in order for me to improve the source load performance and at the same to optimize the vcenter load during the backup job

Thanks and Regards

Vitaliy S.
Product Manager
Posts: 23010
Liked: 1560 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How to improve source load: 99%

Post by Vitaliy S. » Jun 18, 2014 9:48 am

Hi Nufael,

If you storage cannot provide more performance and backup job cannot read data any faster, the only way to improve this is to upgrade the hardware. However, there is a couple of things that I would like to check in your setup:

1. Can you please confirm you're using Direct SAN backup mode? Can be done via job sessions, just search for [san] tag there.
2. Is there any special reason you're using Veeam B&R v6.5 that does not have full support for vSphere 5.5?
3. Was it a full or incremental backup job run?

Thank you!

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 18, 2014 10:08 am

Hi Vitaly,

Appreciate your response

To answer your question.

1. Automatic mode with enable failover to network mode. If Im not mistaken the SAN access mode is not supported with NFS storage (cifs repository)

2. No specific reason and I believe I'm not aware on the weaknees on the integration between Vsphere 5.5 and Veeam Backup and Replication v6.5. Can you advice me on the suitable version that can suit with vsphere 5.5?

3. Yes, full forward incremental backup job

Thanks and Regards

Vitaliy S.
Product Manager
Posts: 23010
Liked: 1560 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How to improve source load: 99%

Post by Vitaliy S. » Jun 18, 2014 10:21 am

Nufael,

1. I was not talking about target, but rather VM source datastore. Please take a look a this topic for more details > Direct SAN Access Mode. Try to configure direct SAN mode in order to have better job performance.
2. Here you go > [ALL VERSIONS] Current version is 7.0 Patch #4 (7.0.0.871)
3. Actually I was asking what job session was that. Was it a full/initial job run?

Thanks!

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 18, 2014 12:52 pm

1. By referring to the statistic and report via the job session, there is no [san] tag. If my step is incorrect please help to lead in the correct way.

Referring to your advice, in order to get better job performance, the best way is to configure direct SAN mode. My environment is, no FC HBA and iSCSI target available at a moment. I'm using CIFS for the target location (EMC Unified Storage). Any best advice?

2. Thanks, appreciated

3. Full

Thanks

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How to improve source load: 99%

Post by foggy » Jun 18, 2014 12:57 pm

nufael wrote:1. By referring to the statistic and report via the job session, there is no [san] tag.
That means that network backup mode is used. Do you see the VMFS volumes in the Disk Management snap-in on the Veeam B&R server?
nufael wrote:Referring to your advice, in order to get better job performance, the best way is to configure direct SAN mode. My environment is, no FC HBA and iSCSI target available at a moment. I'm using CIFS for the target location (EMC Unified Storage). Any best advice?
Vitaliy's advice regarding direct SAN mode applies to reading data from the production datastore, not writing them to the backup target.

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 18, 2014 1:49 pm

Hi Foggy,

1. No. Do you think I need to have FC HBA installed at Veeam Backup server or prepare iSCSI target in order to boost the backup performance? Would it also improve on source load?

2. Meaning that I need to check at the Vsphere setting on how they reading data from datastore which located at EMC unified storage?

regards

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How to improve source load: 99%

Post by foggy » Jun 18, 2014 2:20 pm

Basically, everything you need to enable direct SAN mode is described in the FAQ topic referred to above:
Q: How to configure the new backup proxy server for direct SAN access?
A: Disable automount (done automatically when backup proxy server components are installed), install and configure SAN HBA card (or iSCSI Software Initiator), give the backup proxy server machine access and read permission on VMFS LUNs (if applicable to your SAN), and ensure that you are seeing your VMFS volumes in the Windows Disk Management snap-in. Refer to the following thread for more information specific to certain SAN makes and models: Configuring new VEEAM Backup Server for SAN.

Vitaliy S.
Product Manager
Posts: 23010
Liked: 1560 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How to improve source load: 99%

Post by Vitaliy S. » Jun 18, 2014 3:51 pm

Does you production storage have FC or iSCSI connection? If one of these, then you can configure Direct SAN mode and read data more efficiently. If there is no opportunity to do that, then I would suggest switching to HotAdd backup mode. This will require virtual proxy servers on the hosts/clusters you're backing up VMs from.

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 19, 2014 6:28 am

Production storage for Veeam backup repository or production storage for VM datastore? For VM datastore we have FC connection to ESXi host but for VBR repository we dont have iFC or iSCSI connection.

Another question, referring to your advice earlier to upgrade VBR v6.5 to 7 due to fully support for vSphere 5.5. Is there any major impact if I'm still using v6.5 with vSphere 5.5 rather than v7?

thanks

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 19, 2014 7:47 am

I think i understand your statement wrongly. This should clear the cloud. In my proxy environment, there is no FC or iSCSI connection to production storage. FC and iSCSI connection are used to communicate ESXi host with storage datastore (resides in EMC) only. Proxy server only used ethernet connection to retrieve data from source.

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How to improve source load: 99%

Post by foggy » Jun 19, 2014 9:05 am

If there's no ability to use direct SAN mode, then consider deploying the virtual proxy to be able to use hotadd for data retrieval.

Vitaliy S.
Product Manager
Posts: 23010
Liked: 1560 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How to improve source load: 99%

Post by Vitaliy S. » Jun 19, 2014 9:29 am

nufael wrote:Another question, referring to your advice earlier to upgrade VBR v6.5 to 7 due to fully support for vSphere 5.5. Is there any major impact if I'm still using v6.5 with vSphere 5.5 rather than v7
The major impact is the following - if you run into any issues with this configuration, our support team will not be able to help you, as VBR v6.5 does not officially support vSphere 5.5. Thanks!

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: How to improve source load: 99%

Post by foggy » Jun 19, 2014 9:34 am

You can also find some usefull information in this thread.

Vitaliy S.
Product Manager
Posts: 23010
Liked: 1560 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How to improve source load: 99%

Post by Vitaliy S. » Jun 19, 2014 9:40 am

nufael wrote:FC and iSCSI connection are used to communicate ESXi host with storage datastore (resides in EMC) only. Proxy server only used ethernet connection to retrieve data from source.
Just to make sure we are on the same page, you have production datastores that are connected via iSCSI and some via FC? If you don't have any spare HBA adapters for FC storage, then proceed with hotadd backup mode. As to the iSCSI connected datastores, then use Windows iSCSI initiator to present LUNs to the proxy server and then use direct SAN mode to retrieve VM data from the storage.

Here is a link on how to configure iSCSI connection from the proxy to the storage > http://www.jpaul.me/?p=334

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 20, 2014 3:21 am

Ahhh, sorry my bad. My production datastore are connected to ESXi host via FC ONLY.

My proxy server is a physical server (not a VM server thus no connection to production datastore via FC).

Correct me if I'm wrong, I can't use Direct SAN backup mode since FC or iSCSI connection are not available in between proxy and production datastore.

Thanks

dellock6
Veeam Software
Posts: 5734
Liked: 1626 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: How to improve source load: 99%

Post by dellock6 » Jun 20, 2014 6:15 am

Correct, in order to use DirectSAN the proxy needs to be connected into the same storage fabric as the ESXi servers, regardless it's fc or iscsi.
you have two options:
- install an FC hba into the physical proxy and use DirectSAN
- add a virtual proxy running inside the same vSphere cluster and use hotadd mode

Oherwise you are only left with network mode.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

nufael
Influencer
Posts: 24
Liked: never
Joined: Jun 18, 2014 8:06 am
Full Name: nufael
Contact:

Re: How to improve source load: 99%

Post by nufael » Jun 20, 2014 6:51 am

In my case, I've multiple ESXi host. Do I need to add multiple virtual proxy to every of each host?

Another thing to note is that, All VMs in ESXi host are connected to datastore using FC connection. If I add virtual proxy it will obviously running in ESXi host environment and use FC connection to production datastore, can it also consider use Direct SAN access mode?

Thanks

dellock6
Veeam Software
Posts: 5734
Liked: 1626 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: How to improve source load: 99%

Post by dellock6 » Jun 20, 2014 8:20 am

No, if one ESXi can see the shared FC storage, only one proxy is enough. You need a local proxy only if you have local datastore and you still want to use HotAdd.

No, the FC connection is in regards of ESXi and FC Storage Array. Virtual proxy will use HotAdd.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 14 guests