Writing VEEAM Agent backups to tape is slow

Everything about backing up to tape

Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Fri Sep 08, 2017 2:32 pm

System software/versions: Windows Server 2012 R2, VEEAM 9.5 Update 2

Server hardware:
HP DL380 G7, 2 X Xeon E5-2960 v2 (20 total cores @ 3.0GHz), 192 GB RAM

Storage:
Repository: Promise VESS R2600xid, 220 TB (formatted), 10 GB/s iSCSI network
Tape drive/media: HP 1x8 G2 Autoloader (SAS), LTO-7, with HP LTO-7 tapes

We have set it all up and are getting good performance out of the backups of our VMs to tape (about 270 MB/s), but the backups produced by VEEAM Agents on a couple of physical servers is really slow (about 40 MB/s). The bottleneck for those tasks is reported as: Source 91% > Proxy 0% > Network 14% > Target 25%. There is nothing else accessing that source at the time the tape job is running. In fact, there isn't anything happening on the server either (it is after business hours, and the other backup prep/job starts 2 hours later).

All the other tasks in the tape job report something like this: Source 12% > Proxy 6% > Network 24% > Target 99%. This is what I wold expect.

I don't understand why the two VEEAM Agent generated files are so much slower. I can't find any way to change anything about how the tape job processes those VEEAM Agent related backups either. What options do I have to speed those tasks up?

Thanks!
Clint
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby PTide » Sun Sep 10, 2017 8:00 pm

Hi,

Is that Agent for Linux of for Windows that you use? Also what filesystem is being used on the repository?

Thanks
PTide
Veeam Software
 
Posts: 3252
Liked: 273 times
Joined: Tue May 19, 2015 1:46 pm

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Mon Sep 11, 2017 12:13 pm

It is all Windows. The host being backed up is Windows Server 2008 R2. The Repository is formatted NTFS, mounted over iSCSI (physical network connection on the VEEAM server is an Emulex 10 GB/s OCe14102-U). What gets me is that all my VM backup jobs get the expected throughput. The same job just went last night, but skipped the agent backups from physical servers as they had full backups going and no updates for tapes. It reported writing at 289 MB/s. But when it has to write the agent output to tape, it crawls through them. then it zooms through the VM backups.
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby Dima P. » Mon Sep 11, 2017 1:03 pm

Hello Clint,

Are you using encryption both in agent backup jobs and in tape jobs?
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Mon Sep 11, 2017 8:09 pm

We are not using encryption in any jobs.
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Tue Sep 12, 2017 12:31 pm

Just noticed one more detail... after an active full backup from the agent, the tape write speed for that file is as fast as expected. So the performance impact is only when it is processing our weekday incremental backups. Looking closely at the job results in the UI, it saw a VBK to write to tape. This is different than what is usually there. The job sees usually indicates it is processing a "VBK map" instead of a VBK.
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby PTide » Tue Sep 12, 2017 1:41 pm

So the performance impact is only when it is processing our weekday incremental backups. Looking closely at the job results in the UI, it saw a VBK to write to tape. This is different than what is usually there. The job sees usually indicates it is processing a "VBK map" instead of a VBK.
So, you say that you get daily VBK to tape, is that correct? If so then may I ask you what are the days that you've specified for Full backup?

Thank you
PTide
Veeam Software
 
Posts: 3252
Liked: 273 times
Joined: Tue May 19, 2015 1:46 pm

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Tue Sep 12, 2017 4:17 pm

Full backups happen on the weekends... Tape jobs run monday through friday, so on Mondays we get the previous weekend's active full backup onto tape.
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby einhirn » Tue Oct 10, 2017 7:00 am

I'm seeing this problem, too, can't say that it happens every time a VBK-Map has to be written, can't say that it only occurs with VBK-Maps... Anyway, I've just opened a case to have the issue investigated: #02339506
einhirn
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Mon Feb 02, 2015 1:51 pm

Re: Writing VEEAM Agent backups to tape is slow

Veeam Logoby ccone@vascorltd.com » Mon Oct 23, 2017 6:55 pm

I think upgrading the Veeam agent on the remote servers seems to have resolved this for us. But I'm not sure why.
ccone@vascorltd.com
Influencer
 
Posts: 11
Liked: 1 time
Joined: Fri Sep 08, 2017 1:47 pm
Full Name: Clint Cone


Return to Tape



Who is online

Users browsing this forum: No registered users and 1 guest