v9 Question about parallel tape processing

[ekisner]

Question... I see you're implementing GFS rotations. This makes me VERY happy.

Since I see that the Veeam devs have given some love to the tape aspects of Veeam, I thought I might inquire as to multiple drive processing - a single library with multiple drives in it.

At present we have a library with a single drive, and the disk to tape job takes about 5 and a half days to run. This is not ideal! Through browsing the forums I see that as of V8, multiple drives must be implemented with multiple media pools and multiple jobs. Which is also not ideal. It can be Frankensteined together but it would require maintenance to make sure that the workaround maintained a balance.

So... is there any support for tape parallelism coming in v9? 2 drives should cut my window to 2 1/2 days, 3 drives even less.

There would likely be FC connectivity.

[Shestakov]

Hello Erik,
Parallel processing between multiple tape drives becomes available in v9. Previously you needed to set up multiple media pools and tape backup jobs, what increased complexity, in v9 it is available out of the box.
Read more about new tape features in v9.

[Dima P.]

Thanks – glad to hear and moreover glad to share your feedback with the team.

In v9 we will have two “levels” of parallel processing available. One is parallel processing between simultaneously running gobs pointed to the same media pool, another is parallel processing between source backup chains. That should bring your performance to the new level – stay tuned for then BETA, and check the blog post Nikita has shared. Cheers!

[ekisner]

That makes me very happy!

Lots of tape love this patch, very excited for v9 release! Thank you both for your followup.

[Shestakov]

Thanks for the kind words, Erik!
I`ll forward them to the developers working on tape features.

[rreed]

Would you mind expanding a little upon the parallel processing of backup chains please? Bear w/ me please as I have just started diving into Veeam's tape capabilities the last couple of days. We have two drives in our library and having them both going at the same time sure would be great. I know some parallelism I would like to see would be a single one-to-many backup where my backup writes to disk while simultaneously writing to tape rather than traditional B2D2T.

Our reason being is that right now we write directly to deduplicating device (EMC DD's and Dell DR's). The tape job has to pull that data from the dedupe device, which we all know to be slow, rehydrating, etc. Yes I know we need to have JBOD as our staging area before archival storage, and it's in the works, we just don't know when, and it won't be the same 84+ TB of production storage we have on tap. Being able to point at least some of our backup jobs straight to tape that won't fit on our staging LZ would really help out.

[Shestakov]

Hello rreed,
Backup-to-tape job is supposed to be a secondary job to make a copy of the VMs backups and store it on tapes. I see your point but it can only copy existing restore points to the tape media.
In a case of copy process skipping the dedupe device, you would need to make an additional backup of the VMs, what would take even more time and resources.
Thanks!

[veremin]

Would you mind expanding a little upon the parallel processing of backup chains please?

In version 9 we will introduce new feature called per-virtual machine (VM) backup file chains, meaning that each VM selected as a source for backup job will have its own set of full and incremental files.

If you select such job (containing multiple chains) as a source for backup to tape job, tape job will be able to process those chains in parallel, distributing the workload among available drives.

Thanks.

[rreed]

This is so very incredibly full of win, guys. Excellent, excellent job, looking VERY forward to this and other new features!! Many thanks!

Question on the per-virtual machine backup file chains, right now if we need to move backup files (say, a given job) to another repository and point the job to the new location (works great); will we be able to move only some of the VMs' backup files of the per-VM chain? In short, split the VM's out of a given job, if that makes sense? Or will it still be the same way were we have to move everything; it's just in many mulitple files vs. just the chains only. If that also makes sense. For cases where a given repository needs just a bit of space freed up but I don't have time/landing zone to move it all.

[Shestakov]

Here is another top-feature, Scale-Out Backup Repository. With the feature, you can use several repositories, logically combined to a single one, as a target for your job.