Discussions specific to the VMware vSphere hypervisor
Post Reply
astrumbale
Novice
Posts: 4
Liked: never
Joined: Jan 13, 2014 3:45 pm
Full Name: Alberto Lazzarin
Contact:

Backup target bottleneck

Post by astrumbale »

Good morning to all, i'm alberto from italy, this is my first post in this forum (if i forget something please tell me..).

I've got some issue on my backup:

- backup windows show me that the bottleneck was Target at 88%.
- the job speed is 10 MB/s.
- backup proxy is configured in virtual appliance mode, without failover to lan flag.
- backup proxy is w2k3r2 x64 and it has: 4vcpu 8gb of ram and a RDM repository of 1.95 TB.
- backup proxy and repository are on same machine, and the server ha only veeam backup proxy function.
- resource utilization of veeam proxy was cpu 25% and ram 3.32 GB
- if during backup i try a benchmark on the logical disk repository the result was: read 338.1 MB/s - write 219.8 MB/s (sequential, 1 GB, 9 pass)

i've analyze the throughput of esx path where proxy resides and it was used at 30% maxium, the lan was not used, no error at the end of job.
the total job data:
duration: 57.18 min
processing rate: 10 MB/s
bottleneck: target (source 8% - proxy 18% - network 0% - target 88%)
processed: 142.9 GB
read: 23.3 GB
Transferred: 23.1 GB
Success: 6

my ifrastructure:
veeam 7.0 r2 enterprise
vmware 5.1 u1

have you got any idea about the issue?

many thanks,
alberto

foggy
Veeam Software
Posts: 19285
Liked: 1743 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup terget bottleneck

Post by foggy »

astrumbale wrote:bottleneck: target (source 8% - proxy 18% - network 0% - target 88%)
Alberto, this means that the bottleneck is clearly the write speed to the target storage. What kind of storage do you use?

dellock6
Veeam Software
Posts: 5921
Liked: 1739 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup terget bottleneck

Post by dellock6 »

Sounds like a reversed incremental job too, can you confirm? If that's the case, and if the target storage has not such high performances, this kind of results is somewhat expected. Have a look at this if you want:

http://www.virtualtothecore.com/en/veea ... torage-io/

Also, do not confuse sequential write performances with Veeam backup performances. Especially on reversed incremental, writes are heavily random, so the numbers you are going to obtain could not be the same of that kind of tests.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

astrumbale
Novice
Posts: 4
Liked: never
Joined: Jan 13, 2014 3:45 pm
Full Name: Alberto Lazzarin
Contact:

Re: Backup terget bottleneck

Post by astrumbale »

@foggy,

i've try to do a benchmark during the backup session, backup speed 10 MB/s, storage bench result: read 338.1 MB/s - write 219.8 MB/s
with that result i can exclude a storage problem.
my storage is ibm san volume controller, 3 io group.

@dellock6,

yes we use reversed incremental.
my storage is ibm san volume controller, 3 io group.
the performance test was done during the backup session to exclude storage write bottleneck, i know that the sequential rw of test must not be compared with veeam speed, but was the only way to exclude storage problem.

thanks,
alberto

Gostev
SVP, Product Management
Posts: 26524
Liked: 4155 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup terget bottleneck

Post by Gostev » 1 person likes this post

astrumbale wrote:i know that the sequential rw of test must not be compared with veeam speed, but was the only way to exclude storage problem
Considering that sequential rw test is completely useless for storage diagnostics, how does this exclude storage problem? Try using IOMeter with the following storage profile, this should mimic reversed incremental backup's I/O pattern quite closely.

astrumbale
Novice
Posts: 4
Liked: never
Joined: Jan 13, 2014 3:45 pm
Full Name: Alberto Lazzarin
Contact:

Re: Backup target bottleneck

Post by astrumbale »

test with iometer in progres.. later i can tell you.

attached the volume usage during backyp job:
https://www.dropbox.com/s/zaozienxf26xc ... 58-30.html

astrumbale
Novice
Posts: 4
Liked: never
Joined: Jan 13, 2014 3:45 pm
Full Name: Alberto Lazzarin
Contact:

Re: Backup target bottleneck

Post by astrumbale »

here the iometer result:
https://www.dropbox.com/s/tnbhzg4y72sl0fn/results.csv

here the volume grapsh stat during backup oif yesterday:
https://www.dropbox.com/sh/tvxof1u5im53lyl/5uDQVv7b4y

Post Reply

Who is online

Users browsing this forum: Baidu [Spider] and 22 guests