Discussions specific to tape backups
Post Reply
MurkyBuffalo
Lurker
Posts: 2
Liked: never
Joined: Feb 08, 2016 5:48 pm
Full Name: EJ Pennyman
Contact:

[Feature Request] Parallel tape processing fine tuning

Post by MurkyBuffalo » Oct 24, 2017 7:56 pm

Veeam B&R v9 brought per media set parallel processing, which I use regularly. However, here are some requests for fine tuning the parallel processing setting.

Certain repositories and jobs that would benefit from not being allowed to parallel process. I would use the repository maximum task limit, but here's the situation: the repository maximum does not specify type of activity. I would like to specify how many of each type of activity is allowed, for example:
- Backup to disk and backup to tape at the same time, good.
- Multiple streams to tape from one repository, bad.

Here's why I would like a "Tape streams per job or per repository" setting. I have some repositories that are SSD and can handle multiple read streams and saturate the drives' write speeds. I have other repositories that are slow arrays that start to thrash with multiple read streams, leading to very poor job performance. Maybe a way to do this would be a speed minimum for parallel processing. Meaning one drive must be saturated before adding a second drive. This would help prevent the drives from being hogged by poor performing jobs.

It would even be acceptable to have per job limits. ie, The media pool can use up to 3 tape drives, but each job can only claim 1 or 2 of the drives. This would be useful to allow smaller jobs to have an available drive while larger jobs chug away.

My use case here is that I have several large full backups that run weekly. I would like to allow the smaller backup jobs to run besides these big ones. Both jobs are full, so using a separate incremental pool would not help. I would prefer to keep my weekly jobs in one big pool to make long term tape management easier.

In summary, I'm suggesting more fine tuned parallel tape processing options:
1) Repository simultaneous job limits per job type
2) Setting "Tape streams per job" or "Tape streams per repository"
3) Minimum speed requirement before a job spawns a second stream

einhirn
Enthusiast
Posts: 33
Liked: 3 times
Joined: Feb 02, 2015 1:51 pm
Contact:

Re: [Feature Request] Parallel tape processing fine tuning

Post by einhirn » Nov 03, 2017 8:45 am

Hello,
I second this request and would like to make an amendment, because it concerns the same topic:

To further improve throughput in a job it would be great to sort the backup chains/files by size descending.

That way, when a job has some big and some small files, all available drives will be used most of the time. Currently I see the following behavior: Parallel processing starts with small chains and uses all drives. Then at some point it gets to the few big ones and some tape drives go idle, because there are no more chains to work on, resulting in a reduction of overall throughput and longer job duration.

Thanks for your consideration.

bye
einhirn

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests