Discussions specific to tape backups
sentenza
Novice
Posts: 5
Liked: 3 times
Joined: May 27, 2015 7:41 pm
Full Name: Olivier Tarnus
Contact:

Feature request: parallel tape jobs to same media pool

Post by sentenza » May 27, 2015 8:49 pm

Dear Veeam community,

Here is my "official" feature request: Authorize multiple jobs to request tapes from the same media pool. This comes after weeks of investigation with support (#00850675), and seems the minimal requirement to support "big" tape jobs, ie: more than 12TB in less than 24 hours.

Problem: When a job starts, it locks the media pool and will only release it when finished. Another job that could run on another drive in the same library can not request tapes from the same media pool until first jobs is finished, greatly limiting the throughput to only one drive. The only alternative is to add more media pools, adding far too much complexity and fragmentation in the library.

Why is this needed: As Veeam is not implementing tape multi-streaming (ie: writing the same job to multiple tapes in parallel), the only alternative for "big data volumes" is to run multiple jobs in parallel on individual drives. Managing more jobs already adds complexity, but fragmenting the media pools decrease efficiency dramatically and totally "disconnect" the pools from what they should be (ie: Daily, Weekly, Monthly...).

Expected result: A job should not lock the media pool but only the tape it is currently using. Other jobs should be able to claim another (free) tape in the same pool.

My experience on this: I have totally changed my strategy because of tape issues...

6 month ago, I had 3-4 jobs backing up 400 VMs to 2 big repositories. It was easy to manage and error prone: basically we did backup everything, and were sure no VM was left away. And then we started to implement tape "for real"...

Fast forward 6 months: I have 15 jobs, backing up to 7-8 repositories. We have also added copy jobs and their own repositories as a "buffer", because tape jobs were locking the main repositories thus preventing production backups to run. Everything is now more complex, our storage usage as exploded and it definitely requires more time to manage, but my VM count and volume increased only by 5% and this makes it hard to explain to management... And still can't come up with a viable tape strategy...

We need a solution, as 12TB in 24 hours (basically 1 LTO6 drive full speed) does not seem "that" big in my opinion and set the bar quite low for tape users.

Olivier

justind
Enthusiast
Posts: 32
Liked: 4 times
Joined: Nov 09, 2014 4:20 am
Location: Australia
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by justind » May 28, 2015 1:30 pm

+1 vote from me

Being able to share media pools would be great. Currently I have 18 Media pools that I could reduce down to 3 with shared pools

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » May 28, 2015 2:35 pm

Hello guys,
I can’t provide you any details on the next major version feature set, however, I want to encourage you to wait for it as the issue you described will be resolved with some additional really cool new functionality :wink:

robpare
Lurker
Posts: 1
Liked: never
Joined: Sep 16, 2014 9:57 am
Full Name: Rob
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by robpare » Jun 09, 2015 1:43 pm

+1 from me as well and I look forward to see what comes in the next version.

We've just started setting up tape media pools and have been caught out by media pools being locked when in use by one job. Sentenza captures the problem more perfectly than I think I'd have done.

We can come up with workarounds for this but I'd much rather go back to having a small number of media pools - we might get away with just two for weekly/monthly if the lock happened per tape rather than at the pool level.

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » Jun 09, 2015 2:24 pm

So you would like to point multiple tape jobs to the singe media pool and let this tape jobs run in parallel, right?

gosnells
Influencer
Posts: 12
Liked: 1 time
Joined: Jul 23, 2012 4:28 am
Full Name: Mike Smith
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by gosnells » Jun 12, 2015 7:06 am

+1
being able to fully utilise an LTO6 tape library with two drives is a must. Surely tape support was added for / aimed at enterprise users, and most enterprise users do not muck around with a single drive. Splitting the tape backup jobs into multiple media pools to get concurrency is crazy.

bvrz
Lurker
Posts: 1
Liked: never
Joined: Jun 15, 2015 8:40 am
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by bvrz » Jun 15, 2015 8:52 am

hello guys,

when comes the next major version Feature with these addin?
thanks

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » Jun 17, 2015 4:11 pm

bvrz,
There is no ETA at the moment.

Mike,
Are you ok with a possible tape media overrun?

Gerwin1
Novice
Posts: 4
Liked: never
Joined: Jul 16, 2013 9:34 am
Full Name: Gerwin van de Bunt
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Gerwin1 » Jun 23, 2015 8:48 am

+1 for me to.
I have a tape drive with 3 LTO 6 drives. Now we have 3 media pools. When I need 100 GB additional tape space in media pool 1 and I have space left on a tape in media pool 2. It is a pain in the ass. Most of the time I add an extra tape to the pool wich is insane because I only need 100GB wich is available on media pool 2. It would be an nice faeture to have 1 medial pool with 3 drives (atthaced to 3 tape servers), so that tapes are use more effecient.
I don't have a problem with using seperate tape jobs for each drive.

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » Jun 24, 2015 5:36 pm

Gerwin,
Thanks! Have you considered the ability to run tape jobs simultaneously - one per drive only or multi-streaming whithin one tape job is also a possibility? Thanks

aaron@ARB
Expert
Posts: 121
Liked: 12 times
Joined: Feb 21, 2014 3:12 am
Full Name: Aaron @ ARB
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by aaron@ARB » Jun 25, 2015 10:15 am

+1 from me too.. Its so frustrating to have 2 LTO6 drives and only really able to use one.

but PLEASE put some proper thought into it and give a lot of customisation options in relation to such an option as there ARE times that you want to lock the media pool so jobs will run in serial, but as said above, BIG data jobs would benefit from this functionality

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » Jun 25, 2015 11:12 am

Aaron,
+1
To which one: job multi-streaming or backup files within one job multi-streaming? :D
give a lot of customisation options in relation to such an option
I was thinking about tape jobs and files multistreaming, and settings that would allow to tune the actunal number of drives that could be used for multi-streaming within one media pool. Any other customization suggestions are highly appreciated

aaron@ARB
Expert
Posts: 121
Liked: 12 times
Joined: Feb 21, 2014 3:12 am
Full Name: Aaron @ ARB
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by aaron@ARB » Jun 25, 2015 11:50 am

Make that' +2 then as both are good

I really hate to say this but check out backup execs tape management. They have been doing tapes for eons, they have it working pretty well from THAT perspective. (Runs into my post about tape views too). Multi streaming and multi files are both great ideas. Tape is still something complex to get right and us with auto loaders have them for a reason. I for one have to manage 12-14tb out to tape. It's a struggle, anything that can be done to stream that data out to as many drives as possible is an advantage.

michaelryancook
Expert
Posts: 116
Liked: 14 times
Joined: Nov 26, 2013 6:13 pm
Full Name: Michael Cook
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by michaelryancook » Jun 25, 2015 7:14 pm

I would say having both options would be great. Multiple files to multiple drives within the same job or multiple jobs to the same pool at the same time.

Dima P.
Product Manager
Posts: 9605
Liked: 755 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Feature request: parallel tape jobs to same media pool

Post by Dima P. » Jun 26, 2015 2:25 pm

Thanks guys. Your feedback makes the picture less blurry!

Post Reply

Who is online

Users browsing this forum: Majestic-12 [Bot] and 4 guests