Bootleneck: "Source" without reason

VMware specific discussions

Bootleneck: "Source" without reason

Veeam Logoby Elinet » Tue Oct 06, 2015 4:20 pm

Goodmorning,

I'm Nicola from Elinet.
I've already used veeam as a backup solution in the past.

At the moment i'm facing a problem regarding the bottleneck of bacakups. My backup speed when i start a job is around 12MB/s.

Veeam is installed in a VM inside a Host with ESXi 6.0. I configured all the environment with the Vcenter server. As backup repository i added a CIFS share and an ISCSI disk (for test). All of them are from my NAS.
Veeam's proxy configuration: Virtual Appliance.
I configured NAS, ESXi, and all VMs with jumbo frames at 9000.

If i try to upload or download a large file from my NAS by CIFS, the average speed is 100 MB/s.

The main Datastore of my Virtual environment is conneted by ISCSI. Disk peformance are good between VMs (around 80 MB/s).

I'm not understand why bottleneck still remaining "Source".

What i'm doing wrong?

regards
Elinet
Novice
 
Posts: 4
Liked: never
Joined: Tue Oct 06, 2015 4:09 pm
Full Name: Elinet

Re: Bootleneck: "Source" without reason

Veeam Logoby PTide » Tue Oct 06, 2015 5:01 pm

Hi,

Was it a full backup run or an incremental one? Could you please provide a full bottleneck statistics for full backup and job session log? Also please check if you have any kind of IO throttling configured. Btw, have you checked IO queue length on your storage during the job run?

Thank you.
PTide
Veeam Software
 
Posts: 3022
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: Bootleneck: "Source" without reason

Veeam Logoby dellock6 » Tue Oct 06, 2015 5:09 pm

Also, remember a file copy is a pure sequential read/write activity, almost any backup operation is not, so a direct comparison between the two operations cannot be done.
It's like saying how fast my car can run on a 100km straight line, or trying to be as fast as possible in racetrack with turns...
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5052
Liked: 1332 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Bootleneck: "Source" without reason

Veeam Logoby Elinet » Wed Oct 07, 2015 7:21 am

Thanks all for replying me

dellock6 wrote:Also, remember a file copy is a pure sequential read/write activity, almost any backup operation is not, so a direct comparison between the two operations cannot be done.
It's like saying how fast my car can run on a 100km straight line, or trying to be as fast as possible in racetrack with turns...


Mine was only an example just to have more details. I noticed a huge gap between them so i wanted to write it.

PTide wrote:Hi,
Was it a full backup run or an incremental one? Could you please provide a full bottleneck statistics for full backup and job session log? Also please check if you have any kind of IO throttling configured. Btw, have you checked IO queue length on your storage during the job run?
Thank you.


I did my tests with an Active Full.
i don't have any kind of throttling configured in veeam (I checked the link too)

Here some other options i modified:
- Backup Repository. (CIFS and ISCSI) I removed "Limit Maximum concurrent Task" and "Limit combined data rate to"
- Vmware Proxy. I change transport mode to "Virtual appliance".
- Job Backup. I changed "Storage optimization" to "LAN target" (in my CIFS test. Local Target on my ISCSI Test).

This is what i changed from default settings.

i will post ASAP all the job statistics

Thanks all

Regards
Elinet
Novice
 
Posts: 4
Liked: never
Joined: Tue Oct 06, 2015 4:09 pm
Full Name: Elinet

Re: Bootleneck: "Source" without reason

Veeam Logoby Elinet » Wed Oct 07, 2015 7:54 am

Elinet
Novice
 
Posts: 4
Liked: never
Joined: Tue Oct 06, 2015 4:09 pm
Full Name: Elinet

Re: Bootleneck: "Source" without reason

Veeam Logoby PTide » Wed Oct 07, 2015 10:31 am

Ok, that really seems like slow reads from storage. Have you checked datastore I/O queue depth during backup on your ESXi?
PTide
Veeam Software
 
Posts: 3022
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: Bootleneck: "Source" without reason

Veeam Logoby foggy » Wed Oct 07, 2015 10:36 am

I'd also try network mode, just to compare.
foggy
Veeam Software
 
Posts: 14744
Liked: 1082 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Bootleneck: "Source" without reason

Veeam Logoby Elinet » Wed Oct 07, 2015 4:16 pm

Last updates

i've tried also "network mode" without results.

I made some changes to the Virtual disk of the VM running veeam.

it was configured with "Thick provisoned"
Now i changed with "Thick provisoned Eager zeroed". I also deleted all snapshot of the machine and change disk type to "independent persistent".

...nothing changed :(

maybe i need to do some other tuning?
Elinet
Novice
 
Posts: 4
Liked: never
Joined: Tue Oct 06, 2015 4:09 pm
Full Name: Elinet

Re: Bootleneck: "Source" without reason

Veeam Logoby Gostev » Wed Oct 07, 2015 4:38 pm

Possibly... but we can't troubleshoot storage issues over the forum posts effectively, so please open a support case and have our engineers take a closer look at your deployment.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland


Return to VMware vSphere



Who is online

Users browsing this forum: Google [Bot] and 31 guests