Remote tape library. Slow performance

Everything about backing up to tape

Remote tape library. Slow performance

Veeam Logoby nreutemann » Mon Nov 10, 2014 3:50 pm

I got this "situation",

We got two TS3100 on our backup infrastructure. After we upgrade to v8, the jobs "backup to tape" shows a "network bottleneck", and the throughput is "flat" on 24Mb/s. The same jobs with the v7, has a troughput of 90Mb/s.
One of the library is conected to the Veeam Sever, the other one, on a remote server (the lan between both servers are 1Gbps).

Any idea?

Thanks in advance,

Nicolas.

Support case: ID#00674598
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Re: Backup to tape - Network bottleneck

Veeam Logoby Dima P. » Mon Nov 10, 2014 4:27 pm

Nicolas,

From the top of my head - could you check if both tape servers are not listed in the global network traffic rules or in local network traffic rules (go to tape infrastructure, select servers right click server > properties and go to Traffic step)? Thank you.
Dima P.
Veeam Software
 
Posts: 6231
Liked: 439 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Backup to tape - Network bottleneck

Veeam Logoby nreutemann » Mon Nov 10, 2014 4:37 pm

Hi,

Now, yes, I add the subnet to "global network traffic rules".
I set up the throttling to 9999Mb/s.
I do this because that was the 1st thing I think "maybe, there is a rule wich constrains the traffic", but no, after the setup, the situation remains the same.

Thanks!

Nicolas.

(Im from Argentina, thats why my english is so poor, sorry).
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Re: Backup to tape - Network bottleneck

Veeam Logoby nreutemann » Mon Nov 10, 2014 6:24 pm

Now, I delete all the rules about throttling, and the performance still remains the same (same numbers like before I create the throttling rules.)

The weird part of this, both jobs has the same performance, the one that are writing to the locally attached library as the another one, attached to the remote server.
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Re: Backup to tape - Network bottleneck

Veeam Logoby mpozar » Tue Nov 11, 2014 12:20 am

After our Upgrade from V7 to V8 we saw the same issue and found that in our environment; we run Reverse Incremental Backups, the Tape Backup job was processing a Synthetic Full on to the tape and had created a .vsb file and was processing the .VBK and some of our .VRB.

We could not find a way to turn of the creation of the Full on Tape, there is only an option to schedule. We then changed the schedule from the Default of doing it every Saturday to the Last Sunday of December.

When we re-ran the Tape Backup Job we found it was no longer creating a Full and the backup speed returned to about what it was previously but we now have another issue where the Tape Backup Jobs finished with an error message:

MediaPool not found (id: 00000000-0000-0000-0000-000000000000)

We now have a case open with Veeam Support regarding the above error and also trying to get an explanation of the changes to the Tape Backup Process.

As an update, we have managed to get the Tape Backup Job to work successfully by also selection to backup the Incremental; note that we a re running Reverse Incremental and the Veeam documentation makes no mention that I could find that Veeam supports backing up .VRB, and found that the Job now also backed up the .VRB files and then the Job completed without the above error.

Case # 00671757


Have FUN!
Michael Pozar
mpozar
Enthusiast
 
Posts: 30
Liked: never
Joined: Sun Jan 01, 2006 1:01 am

Re: Backup to tape - Network bottleneck

Veeam Logoby v.Eremin » Tue Nov 11, 2014 8:42 am

Hi all,

Currently, we're trying to reproduce the reported issues (network bottleneck and throttling rules). I believe we will update the topic, once we have more information.

As to backup to tape job unexpectedly copying .vrb files, we're already aware of this behavior and have created a private fix. Kindly, contact support team in order to get it.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13255
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

V2 - Remote tape library. Slow performance

Veeam Logoby frinnst » Tue Nov 11, 2014 8:44 am

Hi!

So we upgraded to V8 as soon as it became available for the improved Tape support (yay!). We are, however, experiencing some *slow* performance - 4,6MB/s (Bottleneck: Target).
As mentioned in the topic we are using the remote library feature. The Veeam server is installed on a VM (2012r2) and the Tape hardware (HP 1x8 G2 Autoldr) is connected to a physical 2008r2 server.

So the performance is pretty poor. Any suggestions on how to debug this? We could move to the pysical server if that would improve things but i'd rather not if the performance is the same.
frinnst
Influencer
 
Posts: 15
Liked: never
Joined: Mon Aug 26, 2013 1:32 pm
Full Name: Fredrik

Re: V2 - Remote tape library. Slow performance

Veeam Logoby v.Eremin » Tue Nov 11, 2014 8:50 am

Fredrik,

We are, however, experiencing some *slow* performance - 4,6MB/s (Bottleneck: Target).
As mentioned in the topic we are using the remote library feature. The Veeam server is installed on a VM (2012r2) and the Tape hardware (HP 1x8 G2 Autoldr) is connected to a physical 2008r2 server.

So the performance is pretty poor. Any suggestions on how to debug this? We could move to the pysical server if that would improve things but i'd rather not if the performance is the same.


What type of tape job are you using? Backups to tape or files to tape one? What is selected as a source for it? Couple of big backup files or multiple small files? What performance have you had previously? Can you also provide full bottleneck statistics (the one with percent distribution). Thanks.
v.Eremin
Veeam Software
 
Posts: 13255
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Backup to tape - Network bottleneck

Veeam Logoby nreutemann » Tue Nov 11, 2014 9:18 am

Hi, after read your post, I check the log of the jobs and found the same behaviour. The job has queued the .vbk, .vib and .vrb to backup to tape.

cya

Nicolas.
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Re: Backup to tape - Network bottleneck

Veeam Logoby v.Eremin » Tue Nov 11, 2014 9:23 am

Hi Nicolas,

As mentioned above, kindly, open a ticket with our support team and you will be provided with a private fix soon. It resolves the issue with backup to tape job copying .vrb files.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13255
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: V2 - Remote tape library. Slow performance

Veeam Logoby frinnst » Tue Nov 11, 2014 9:49 am

Can you also provide full bottleneck statistics (the one with percent distribution).


We are using "Backup to tape".

Bottleneck:
Source: 0%
Proxy: 15%
Network: 0%
Target: 99%
frinnst
Influencer
 
Posts: 15
Liked: never
Joined: Mon Aug 26, 2013 1:32 pm
Full Name: Fredrik

Re: Backup to tape - Network bottleneck

Veeam Logoby nreutemann » Tue Nov 11, 2014 10:26 am

v.Eremin wrote:Hi, Nicolas,

As mentioned above, kindly, open a ticket with our support team and you will be provided with a private fix soon. It resolves the issue with backup to tape job copying .vrb files.

Thanks.


I already created an Support ticket, the case ID is on the first post of this thread.

Support case: ID#00674598

Best regards,

Nicolas.
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Re: V2 - Remote tape library. Slow performance

Veeam Logoby frinnst » Tue Nov 11, 2014 10:34 am

What type of tape job are you using? Backups to tape or files to tape one? What is selected as a source for it? Couple of big backup files or multiple small files?


We are testing a simple job that has one full backup (257gb) and two diff (5gb, 9gb)
frinnst
Influencer
 
Posts: 15
Liked: never
Joined: Mon Aug 26, 2013 1:32 pm
Full Name: Fredrik

Re: Backup to tape - Network bottleneck

Veeam Logoby v.Eremin » Tue Nov 11, 2014 11:32 am

Nicolas,
We've just sent a private fix to your support engineer. I believe he will provide you with it soon. Thanks.
v.Eremin
Veeam Software
 
Posts: 13255
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Backup to tape - Network bottleneck

Veeam Logoby nreutemann » Tue Nov 11, 2014 11:41 am

v.Eremin wrote:We've just sent a private fix to your support engineer. I believe he will provide you with it soon. Thanks.


Hi Vladimir, Im testing now the fix.

Fix the .vrb problem, but the bottleneck remains, the speed is 25Mb/s

Best regards.

Nicolas.
nreutemann
Enthusiast
 
Posts: 47
Liked: 6 times
Joined: Tue Mar 06, 2012 11:45 pm
Full Name: Nicolas Reutemann

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 4 guests