Comprehensive data protection for all workloads
Post Reply
dvdrbc
Influencer
Posts: 12
Liked: 3 times
Joined: Jun 17, 2015 2:43 am
Full Name: David Rubic
Contact:

Improving processing rate

Post by dvdrbc »

Hi,

This is our current setup:

Physical Veeam B&R Server - Backups are stored here
2 x Intel Xeon CPU E5-2650 v2 @ 2.60GHz
64GB RAM
16TB Storage Volume (7.2K SAS Disks in RAID 6)

3 x ESXi 5.5 Hosts
2 x Intel Xeon CPU E5-2650 v2 @ 2.60GHz - each host
128GB RAM - each host

iSCSI SAN
Nimble CS215

DR Site
2x ESXi 5.5 Hosts with Local Storage

-------------------------------------------

Now that thats out of the way:
We run forward incremental backup jobs every night with an active full on Sundays
We replicate selected VMs hourly to our DR site which is across the road from our building connected via fibre

My problem is that every now and then a backup job will have a processing rate at approx 100MB/s (which seems to be the highest I've seen it go rarely) but anywhere from 10MB/s to 30MB/s is usually standard, with the Source always being the bottleneck

Replication is the same occasionally a job will replicate with a processing rate of 66MB/s (which seems to be the highest I've seen it go rarely) but generally it is 10MB/s to 30MB/s, same as the backup jobs with the Source always being the bottleneck

To me this just doesn't seem right and I feel it should be a lot faster, is there anything I can do to improve how quickly backups and replication jobs are completed?
foggy
Veeam Software
Posts: 21070
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Improving processing rate

Post by foggy »

David, please look at the job session log, select the particular VM to the left, and look for the transport mode tag reported at the proxy selection step (one of the following: [nbd], [san], [hotadd]). What transport mode is being used by your jobs?
dvdrbc
Influencer
Posts: 12
Liked: 3 times
Joined: Jun 17, 2015 2:43 am
Full Name: David Rubic
Contact:

Re: Improving processing rate

Post by dvdrbc »

Hi Foggy,

For the replication jobs: it uses [san] for source proxy and [nbd] for target proxy

Image

For the backup jobs [san] is used

Image
foggy
Veeam Software
Posts: 21070
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Improving processing rate

Post by foggy »

Have you reviewed the tips for improving iSCSI network throughput that are also mentioned by the link above? You can also try hotadd to see if it can perform faster.
dvdrbc
Influencer
Posts: 12
Liked: 3 times
Joined: Jun 17, 2015 2:43 am
Full Name: David Rubic
Contact:

Re: Improving processing rate

Post by dvdrbc »

I went through this link http://forums.veeam.com/veeam-backup-re ... t4093.html and have applied the suggested changes and have noticed no notable difference

Will hot add make much of a different when using direct san?
foggy
Veeam Software
Posts: 21070
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Improving processing rate

Post by foggy »

You can check, since with hotadd data has another path (via ESXi storage stack).
rborhara
Enthusiast
Posts: 25
Liked: 1 time
Joined: Mar 14, 2015 1:07 pm
Full Name: rakesh borhara
Contact:

Re: Improving processing rate

Post by rborhara »

I have a similiar issue logged with Veeam for replication.
If i run a backup job at the DR site , i get directSAN speeds when using the local dr veeam proxy, however when a replication is running its always NBD using the proxy at the local dr site
Veeam guide mentions that replication is via DirectSAN for the first full backup and for incrementals its always NBD.

However the ticket ive raised i still cannot get it to use DirectSAN on the first replication job.

Regards your backup speed i use ISCSI, and have setup the ISCSI iniators to use the dedicated LAN for direct access to my backup storage I get around 130mb/s.
Are you disks capable of handling the faster throuput?
foggy
Veeam Software
Posts: 21070
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Improving processing rate

Post by foggy »

Rakesh, your issue doesn't seem similar. In the OP's case, direct SAN is used where it should, but the bottleneck is source (slow data retrieval speed).
rborhara wrote:Veeam guide mentions that replication is via DirectSAN for the first full backup and for incrementals its always NBD.
This refers to the transport mode used by the target proxy that is populating replica VM data. Do you also mean the target proxy here?
dvdrbc
Influencer
Posts: 12
Liked: 3 times
Joined: Jun 17, 2015 2:43 am
Full Name: David Rubic
Contact:

Re: Improving processing rate

Post by dvdrbc » 2 people like this post

Hi Foggy,

I've been putting up with the crappy processing speeds since September 2015, until last week when I stumbled upon something.

One of the iSCSI NICS in our physical veeam server was set to AUTO but running at 100MBPS!!

After forcing the iSCSI NICS to run at 1GBPS I have noticed a huge change in processing speed using direct SAN. I can now generally have a processing rate of 70-100MB/S. Sometimes it goes even higher than 100MB/S.

Occasionally I will still see poor processing rates of 9-10MB/s which I am yet to work out why, but 80% of the time we are backing up at speeds that I am now happy with.

Thought I was post back in case anyone else encounters a similar issue.
Post Reply

Who is online

Users browsing this forum: Google [Bot], Semrush [Bot] and 134 guests