Feature request: parallel tape jobs to same media pool

Everything about backing up to tape

Feature request: parallel tape jobs to same media pool

Veeam Logoby sentenza » Wed May 27, 2015 8:49 pm

Dear Veeam community,

Here is my "official" feature request: Authorize multiple jobs to request tapes from the same media pool. This comes after weeks of investigation with support (#00850675), and seems the minimal requirement to support "big" tape jobs, ie: more than 12TB in less than 24 hours.

Problem: When a job starts, it locks the media pool and will only release it when finished. Another job that could run on another drive in the same library can not request tapes from the same media pool until first jobs is finished, greatly limiting the throughput to only one drive. The only alternative is to add more media pools, adding far too much complexity and fragmentation in the library.

Why is this needed: As Veeam is not implementing tape multi-streaming (ie: writing the same job to multiple tapes in parallel), the only alternative for "big data volumes" is to run multiple jobs in parallel on individual drives. Managing more jobs already adds complexity, but fragmenting the media pools decrease efficiency dramatically and totally "disconnect" the pools from what they should be (ie: Daily, Weekly, Monthly...).

Expected result: A job should not lock the media pool but only the tape it is currently using. Other jobs should be able to claim another (free) tape in the same pool.

My experience on this: I have totally changed my strategy because of tape issues...

6 month ago, I had 3-4 jobs backing up 400 VMs to 2 big repositories. It was easy to manage and error prone: basically we did backup everything, and were sure no VM was left away. And then we started to implement tape "for real"...

Fast forward 6 months: I have 15 jobs, backing up to 7-8 repositories. We have also added copy jobs and their own repositories as a "buffer", because tape jobs were locking the main repositories thus preventing production backups to run. Everything is now more complex, our storage usage as exploded and it definitely requires more time to manage, but my VM count and volume increased only by 5% and this makes it hard to explain to management... And still can't come up with a viable tape strategy...

We need a solution, as 12TB in 24 hours (basically 1 LTO6 drive full speed) does not seem "that" big in my opinion and set the bar quite low for tape users.

Olivier
sentenza
Novice
 
Posts: 5
Liked: 3 times
Joined: Wed May 27, 2015 7:41 pm
Full Name: Olivier Tarnus

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby justind » Thu May 28, 2015 1:30 pm

+1 vote from me

Being able to share media pools would be great. Currently I have 18 Media pools that I could reduce down to 3 with shared pools
justind
Enthusiast
 
Posts: 29
Liked: 4 times
Joined: Sun Nov 09, 2014 4:20 am
Location: Australia

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Thu May 28, 2015 2:35 pm

Hello guys,
I can’t provide you any details on the next major version feature set, however, I want to encourage you to wait for it as the issue you described will be resolved with some additional really cool new functionality :wink:
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby robpare » Tue Jun 09, 2015 1:43 pm

+1 from me as well and I look forward to see what comes in the next version.

We've just started setting up tape media pools and have been caught out by media pools being locked when in use by one job. Sentenza captures the problem more perfectly than I think I'd have done.

We can come up with workarounds for this but I'd much rather go back to having a small number of media pools - we might get away with just two for weekly/monthly if the lock happened per tape rather than at the pool level.
robpare
Lurker
 
Posts: 1
Liked: never
Joined: Tue Sep 16, 2014 9:57 am
Full Name: Rob

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Tue Jun 09, 2015 2:24 pm

So you would like to point multiple tape jobs to the singe media pool and let this tape jobs run in parallel, right?
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby gosnells » Fri Jun 12, 2015 7:06 am

+1
being able to fully utilise an LTO6 tape library with two drives is a must. Surely tape support was added for / aimed at enterprise users, and most enterprise users do not muck around with a single drive. Splitting the tape backup jobs into multiple media pools to get concurrency is crazy.
gosnells
Novice
 
Posts: 8
Liked: 1 time
Joined: Mon Jul 23, 2012 4:28 am
Full Name: Mike Smith

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby bvrz » Mon Jun 15, 2015 8:52 am

hello guys,

when comes the next major version Feature with these addin?
thanks
bvrz
Lurker
 
Posts: 1
Liked: never
Joined: Mon Jun 15, 2015 8:40 am

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Wed Jun 17, 2015 4:11 pm

bvrz,
There is no ETA at the moment.

Mike,
Are you ok with a possible tape media overrun?
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Gerwin1 » Tue Jun 23, 2015 8:48 am

+1 for me to.
I have a tape drive with 3 LTO 6 drives. Now we have 3 media pools. When I need 100 GB additional tape space in media pool 1 and I have space left on a tape in media pool 2. It is a pain in the ass. Most of the time I add an extra tape to the pool wich is insane because I only need 100GB wich is available on media pool 2. It would be an nice faeture to have 1 medial pool with 3 drives (atthaced to 3 tape servers), so that tapes are use more effecient.
I don't have a problem with using seperate tape jobs for each drive.
Gerwin1
Novice
 
Posts: 4
Liked: never
Joined: Tue Jul 16, 2013 9:34 am
Full Name: Gerwin van de Bunt

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Wed Jun 24, 2015 5:36 pm

Gerwin,
Thanks! Have you considered the ability to run tape jobs simultaneously - one per drive only or multi-streaming whithin one tape job is also a possibility? Thanks
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby aaron@ARB » Thu Jun 25, 2015 10:15 am

+1 from me too.. Its so frustrating to have 2 LTO6 drives and only really able to use one.

but PLEASE put some proper thought into it and give a lot of customisation options in relation to such an option as there ARE times that you want to lock the media pool so jobs will run in serial, but as said above, BIG data jobs would benefit from this functionality
aaron@ARB
Expert
 
Posts: 112
Liked: 12 times
Joined: Fri Feb 21, 2014 3:12 am
Full Name: Aaron @ ARB

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Thu Jun 25, 2015 11:12 am

Aaron,
+1

To which one: job multi-streaming or backup files within one job multi-streaming? :D
give a lot of customisation options in relation to such an option

I was thinking about tape jobs and files multistreaming, and settings that would allow to tune the actunal number of drives that could be used for multi-streaming within one media pool. Any other customization suggestions are highly appreciated
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby aaron@ARB » Thu Jun 25, 2015 11:50 am

Make that' +2 then as both are good

I really hate to say this but check out backup execs tape management. They have been doing tapes for eons, they have it working pretty well from THAT perspective. (Runs into my post about tape views too). Multi streaming and multi files are both great ideas. Tape is still something complex to get right and us with auto loaders have them for a reason. I for one have to manage 12-14tb out to tape. It's a struggle, anything that can be done to stream that data out to as many drives as possible is an advantage.
aaron@ARB
Expert
 
Posts: 112
Liked: 12 times
Joined: Fri Feb 21, 2014 3:12 am
Full Name: Aaron @ ARB

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby michaelryancook » Thu Jun 25, 2015 7:14 pm

I would say having both options would be great. Multiple files to multiple drives within the same job or multiple jobs to the same pool at the same time.
michaelryancook
Expert
 
Posts: 113
Liked: 14 times
Joined: Tue Nov 26, 2013 6:13 pm
Full Name: Michael Cook

Re: Feature request: parallel tape jobs to same media pool

Veeam Logoby Dima P. » Fri Jun 26, 2015 2:25 pm

Thanks guys. Your feedback makes the picture less blurry!
Dima P.
Veeam Software
 
Posts: 6263
Liked: 442 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 7 guests