Files to tape is painfully slow

Everything about backing up to tape

Re: Files to tape is painfully slow

Veeam Logoby richhartdr » Wed Sep 11, 2013 6:12 pm

LOL, guess I'll quit working on my powershell script to get around the files2tape not supporting folder exclusions...

viewtopic.php?f=29&t=18137&p=88324#p88324

...since ~1MB/s is obviously not doable in production...looks like I'm waiting for the next version <fingers crossed>.

PS - Veeam, please get us current documentation for v7 and tape support specifically...this trial and error powershelling is not fun.
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: Files to tape is painfully slow

Veeam Logoby Gostev » Wed Sep 11, 2013 8:43 pm

foggy wrote:you are copying more than 3 million

Wow o.O, keep in mind we recommend no more than 10000 files per job with 7.0, this is explained right in the dashboard displayed when you click on Tape node. Remember that Veeam tape backup engine is currently optimized for moving small number of large files (such as VM backup files), rather than millions of small files.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Files to tape is painfully slow

Veeam Logoby richhartdr » Wed Sep 11, 2013 8:53 pm

Gostev wrote:Remember that Veeam tape backup engine is currently optimized for moving few large files (such as VM backups) to tape, rather than millions of small files.

Would you go as far as to say, Veeam's not optimized to handle small files at all? I have much less (under 1000) files and my performance is no different (<1Mb/s). Yet, similarly, .vbk backups and my backup to tape jobs cruise along at 40MB/s...not great, but that's my tape hardware limitation.
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: Files to tape is painfully slow

Veeam Logoby Gostev » Wed Sep 11, 2013 9:40 pm

That definitely is correct statement too, but I guess my main point there was that we did not even test the product with millions of files, as again, the primary use case for this initial release of tape support is very different. For example, I don't know if our configuration database queries will scale to millions of files, etc.

Remember, v7 is 1.0 release of tape support aimed at providing ability to copy VM backups to tape. File to Tape is merely a "side" functionality that we were able to deliver almost "for free" in terms of R&D. In fact, this functionality is literally free (available in Free Edition).

We'll be improving this part though, as (unexpectedly) there seems to be strong demand... just cannot have everything at once ;)
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Files to tape is painfully slow

Veeam Logoby richhartdr » Wed Sep 11, 2013 9:52 pm 2 people like this post

<thumbs up> :) files to tape...got my support guys, I'd LOVE to get to ONE backup software, bye bye Symantec! :)
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: Files to tape is painfully slow

Veeam Logoby stjg@sams.ac.uk » Sat Sep 14, 2013 1:03 pm

Yeah, experiencing the same thing - lots of small files just make a Tape job very slow. It would be nice if this could be fixed.

However I am having to use the free version on my archive server (which has the SCSI card and tape libraries attached) just to take advantage of this: my licenced production veeam software is on a Hyper-V virtual machine that uses a backup proxy that has a direct link to the SAN and stores the actual backup files locally : the Veeam machine can't have a directly attached tape library: it would be very useful if Tape jobs could be done through a backup proxy too.
stjg@sams.ac.uk
Novice
 
Posts: 3
Liked: never
Joined: Tue Mar 05, 2013 1:46 pm
Full Name: Steve Gontarek

Re: Files to tape is painfully slow

Veeam Logoby Gostev » Sun Sep 15, 2013 9:26 pm

stjg@sams.ac.uk wrote:it would be very useful if Tape jobs could be done through a backup proxy too

We will support tape devices connected to any Windows server in the short term.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Files to tape is painfully slow

Veeam Logoby joelc67 » Mon Sep 16, 2013 2:58 pm

I have experienced similarly SLOW file to tape performance. We have three large file servers with approximately 100k in files on each environment. If I attempt to backup the file level on each server it goes twelve hours backing up aprox. 120GB of data. If I just backup the VM's themselves it runs a x100 faster. I tried Veeam support. However, the product is not good at backing up 1000s of files. IMHO that makes the file level backups worthless to me. I have had to turn to Backup Exec 2010 (old license) to run file level backups. It is not always practical for me to backup VMs using Veeam 7.0 when archiving files from numerous projects week to week; and having to provide ample space on my backup server for VM's, just so I can later run backup copies to tape for off-site archival purposes is not an efficient use of our server space.

I hope there are plans to change this in a new patch. However, no one could tell me that on Veeam Support. :|
joelc67
Novice
 
Posts: 5
Liked: never
Joined: Mon Sep 16, 2013 2:48 pm
Full Name: Joel Chambers

Re: Files to tape is painfully slow

Veeam Logoby johndoe10110 » Wed Oct 23, 2013 12:56 pm

Just subscribing to this thread (even though there prob won't be any more updates posted in here). I, like many others, look forward to future updates on Files to Tape.
johndoe10110
Influencer
 
Posts: 19
Liked: 5 times
Joined: Wed Oct 23, 2013 12:49 pm
Full Name: John Dooe

[MERGED] Best practice w/ large file quantities & sizes

Veeam Logoby fabian.muecher » Mon Nov 04, 2013 9:39 am

Hello,

I have a client whose file server amounts to 20 TB of productive data. Since they are in advertising I am not able to convince them to scale this down; even archiving everything older than one year would not reduce the size more than 10-15%.
At the moment an agent-based BackupExec job saves the files directly to tape - but since I personally dislike Symantec I would like to switch to Veeam.

A single VM-based backup job seems to be out of the question, since adding an additional storage matching the size of the productivity storage just to be able to use Veeam is difficult to convey.
With the file based job I don't seem to be able to get performance above 5 MB/s when ignoring the 10k file count limit. A performance test (using a filter) with only a couple of hundred iso files gives me about 80-90 MB/s.

Has anyone made similar experiences?
I feel like it seems naive from Veeam to expect file based jobs never to exceed 10.000 files. If they offer this feature, it should be enterprise compatible.

How would you go about backing this up? Split it into multiple jobs to match the available backup disk storage and then replicate to tape? The VMDK are in increments of 2 TB.
Is there any info on Veeam improving file based jobs to handle a realistic amount of files?


Happy for any suggestions.
fabian.muecher
Lurker
 
Posts: 2
Liked: never
Joined: Fri Nov 01, 2013 1:07 pm

Re: Files to tape is painfully slow

Veeam Logoby v.Eremin » Tue Nov 05, 2013 8:20 am

Hi, Fabian, please be aware that this was 1.0 version of tape functionality that was optimized specifically for handling backup files produced by VB&R, thus, there is such a limitation (<10000 files).

However, we consider improving current mechanism in one of the next releases.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Files to tape is painfully slow

Veeam Logoby Hevros » Thu Nov 14, 2013 10:05 am

Gostev wrote:We'll be improving this part though, as (unexpectedly) there seems to be strong demand... just cannot have everything at once ;)


Just to try to give you an understanding of why there is a demand for this function.

A lot of our customers still have one or two physical servers that they cannot or will not make virtual.
We try to push Veeam to these customers but they do not want two backup solutions, they simply say our current backup can do virtual aswell so why change.
now we all know the good things Veeam has to offer but they do not listen when in involves two solutions.

However if we can use Veeam to do a file copy to tape for the last physical file server or the last physical SQL server via maintinance plan backups copied to tape by veeam, then we can offer the customer one solution and they will then buy Veeam.

So when Veeam say it will be addressed in future releases, is that patch release or version release?
Hevros
Service Provider
 
Posts: 25
Liked: 1 time
Joined: Thu Jan 17, 2013 5:09 pm
Full Name: Stephen Seagrave

Re: Files to tape is painfully slow

Veeam Logoby johndoe10110 » Thu Nov 14, 2013 1:38 pm

I imagine it'll be a version release. It's going to be a while, unfortunately.
johndoe10110
Influencer
 
Posts: 19
Liked: 5 times
Joined: Wed Oct 23, 2013 12:49 pm
Full Name: John Dooe

Re: Files to tape is painfully slow

Veeam Logoby v.Eremin » Thu Nov 14, 2013 1:41 pm

Yes, most likely the current file to tape mechanism will be reviewed in the next product release. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Files to tape is painfully slow

Veeam Logoby Gostev » Thu Nov 14, 2013 9:01 pm

Hevros wrote:Just to try to give you an understanding of why there is a demand for this function.

A lot of our customers still have one or two physical servers that they cannot or will not make virtual.
We try to push Veeam to these customers but they do not want two backup solutions, they simply say our current backup can do virtual aswell so why change.
now we all know the good things Veeam has to offer but they do not listen when in involves two solutions.

However if we can use Veeam to do a file copy to tape for the last physical file server or the last physical SQL server via maintinance plan backups copied to tape by veeam, then we can offer the customer one solution and they will then buy Veeam.

Thank you. Here is what I do not get though. What about the same customers who have no tape and no desire to acquire one, which is approximately 66% of small customers like you mention. Why spend time on the solution that will only work for a smaller part of customers?

Secondly, have you considered using Cloud Edition for file level backup with these type of customers instead?
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

PreviousNext

Return to Tape



Who is online

Users browsing this forum: No registered users and 8 guests