Veeam Tapejob needs very long Time

Everything about backing up to tape

Re: Veeam Tapejob needs very long Time

Veeam Logoby gmcoburn » Thu Jun 02, 2016 1:17 am

Thanks for the feedback. I decided to not be lazy yesterday and did my own tests with block sizes ranging from 64KB to 1MB and also found that 64KB block provided me with the highest throughput to tape at 160MBps per tape drive. It also appeared to reduce the shoe-shining. This was just two separate File to Tape jobs, started at the same time, backing up a 42GB file.

Of note the 1MB block size average speed was 130MBps, with the other block sizes speeds being fairly linear up to the 64KB - 160MBps speed.

Oddly there was no difference in speed with Veeam software encryption turned on or off. The backup job does say hardware encryption, but we do not have hardware encryption enabled.

For us there are so many element in play, data\disk read speed, data processing speed for synthetic fulls etc, that I don't think we can expect to approach the full speed of LTO7 drives which is specced at being 300MBps.

For example I am currently running a Backup to Tape job of a single repository (not configured as per-vm) that contains 9 sets of data comprising of 2 Backup (Reverse Incremental) and 7 Backup Copy jobs. The job is set to only back up latest full backups, and synthetic backup built on same day the job has started and with encryption turned on, so in the end the size to go to tape is around 4TB. With parallel processing so 2 drives are in use, at the 50% mark it has taken 3 hours 45 minutes with a Veeam Processing rate of 176MB/s.

This is well under what I had hoped for, as I have another 2 repositories with another 13TB of data that I want to get onto tape within a reasonable time frame.

Any assistance or tips are appreciated.

George
gmcoburn
Influencer
 
Posts: 15
Liked: never
Joined: Thu Aug 28, 2014 5:23 am
Full Name: George Coburn

Re: Veeam Tapejob needs very long Time

Veeam Logoby rreed » Thu Jun 02, 2016 2:15 am

Don't recall if it's been mentioned in your thread, but in several others there are reports of pulling tape backups from across the network being much slower than locally connected HDD's of the physical tape server itself, despite things like 10Gbps infrastructure, fast storage, etc. I understand Veeam to be aware and investigating the issue. It sounds like you've found what others have, that setting block size lower seems to help improve tape performance across the network, counter-intuitively.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)
rreed
Expert
 
Posts: 354
Liked: 72 times
Joined: Tue Jun 30, 2015 6:06 pm

Re: Veeam Tapejob needs very long Time

Veeam Logoby gmcoburn » Thu Jun 02, 2016 3:37 am

Thanks for the heads up rreed.

We do have an infrastructure that is all network connected with the repository server VM connecting to the storage via iscsi and a physical tape drive, all connected at 10Gbps locally. I'll see if I can find the other posts you are referring to so I can get an idea of the issues other are experiencing. I will likely also raise a support ticket with Veeam.

The backup I mention above is now up to 75%, 6 hours and 30 minutes in at a Veeam processing rate of 142MB/s. The job is now on the last backup set in the repository, so just a single drive in operation, but data streaming to that single drive is now down to 70MB/s.
gmcoburn
Influencer
 
Posts: 15
Liked: never
Joined: Thu Aug 28, 2014 5:23 am
Full Name: George Coburn

Re: Veeam Tapejob needs very long Time

Veeam Logoby rreed » Thu Jun 02, 2016 3:44 am

An interesting point in your setup - do I understand your physical tape server is connected to storage via iSCSI rather than just file/CIFS share, etc? Because that's one thing that was suggested to us (we're just using Windows file share to our Windows repositories/CIFS to our dedupe devices). For us the limited SAN repository space we have was labor-prohibitive to connect our tape server to iSCSI infrastructure.

We went round and around w/ support and found the only thing that would help tape across network performance was fiddling w/ the tape drives' block size. Eventually we just gave up and accepted it as-is until they come out w/ a viable solution. As mentioned, last I heard they are aware and working on it but I would still recommend getting a hold of them to help increase visibility, especially since I don't think anyone else is connecting to their storage via iSCSI.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)
rreed
Expert
 
Posts: 354
Liked: 72 times
Joined: Tue Jun 30, 2015 6:06 pm

Re: Veeam Tapejob needs very long Time

Veeam Logoby gmcoburn » Thu Jun 02, 2016 3:59 am

No, we have a repository server which is a Windows 2012 R2 VM, which is connected via VMXNET3 vnics with jumbos frames and RSS enabled, by iscsi to 15TB luns on an EqualLogic san stack. Note that they aren't vmdks but physical luns.

Our tape drive is a separate physical server connected at 10G to the same switch stack as the VM and storage. Its actually in a different room, external to the datacenter.

At this stage I would prefer not to change the repository setup so that the physical tape drive server connects via iSCSi to the storage, as due to other reasons this would require a reorganisation of our national Veeam infrastructure and this would cause "issues".
gmcoburn
Influencer
 
Posts: 15
Liked: never
Joined: Thu Aug 28, 2014 5:23 am
Full Name: George Coburn

Re: Veeam Tapejob needs very long Time

Veeam Logoby rreed » Thu Jun 02, 2016 1:10 pm

Haha, so we're on the same page RE: iSCSI exposure to other servers then. :D RDM's for your repository VM's then? For our Windows repositories they're just vmdk's but it hasn't mattered whether that, CIFS shares, whatever. At the OS level we could pull data from any storage down to the tape server at speed. Backup to tape from tape server at speed. Pull files across the network to tape is slow. Anyways, sounds like a very similar setup to everyone else who's experiencing the same issue. Get you a support ticket going just to make sure nothing is missed, etc. They may do some registry hacks for Veeam to tune it a bit and investigate your tape server's NIC settings.

As a test, stick you a large file or two on your tape server, big as you can fit, and run a test file to tape job from there. See what your speed difference is.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)
rreed
Expert
 
Posts: 354
Liked: 72 times
Joined: Tue Jun 30, 2015 6:06 pm

Previous

Return to Tape



Who is online

Users browsing this forum: No registered users and 6 guests