Tape Support - Feedback / Feature Req

[danatk]

Hi Guys - We love the new tape support in Veeam - we have been testing for 3 weeks and have now gone into production with a working implementation. We really think it's great for a first version, but there are of course a few things we'd like to see improved

No way to identify overwritable media

We can set an overwrite protection period on media sets, but there's no way to see whether a tape is ready for overwrite. Therefore we have no way to easily tell which tapes to retrieve from our offsite store to be overwritten.

Media Pools Locked by single backup job.

Why does the entire media pool get locked by a single job? This means if we have multiple tape drives that we want to run simultaneous jobs, we need multiple media pools for different jobs, or only one drive is used.

Status Display / Duration for Continuous Tape Jobs

It would be nice if jobs had a 'paused' or 'waiting' status when waiting for backup resources. At present we can have multiple jobs running, many of which are waiting for resources, and there is no quick way to see which job is actually running (until the completed percentage increments).

Also, if jobs start and are waiting for resources this increments the duration timer which means we get confusing job durations. For example, we have many continuous tape jobs which start when new backup files from VM jobs appear. These jobs often start when the tape drives are busy with other jobs - at this point they should queue, but not start their timer. When the tape drives are free, the timer should start. This would mean the job duration accurately showed the duration of the job, rather than the time it was queued + duration.

Media Pools are associated with Libraries

During my test period I had to flash the firmware on my tape library and recreate it's partition presented to Veeam. This was then identified as a new library by Veeam and all the media pools (and media) remained associated with the old (now offline in Veeam) library partition. This was OK for me in testing, as I just trashed the old library in Veeam and created new media pools etc. However, if this was a year or more into a production set-up I would have had a major problem identifying and rebuilding all my media pools.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools. Then we can retire libraries, add new libraries etc without issue.

[Dima P.]

Hello danatk,
Thank you for the kind words and outstanding valuable feedback! I am glad and proud that you implemented tape functionality in your production!

No way to identify overwritable media
This functionality is under review right now, and hopefully would be improved in the upcoming versions. Meanwhile, you can create media set's name for different jobs based on your retention – and monitor when tape needs to be loaded under media – offline, where barcode/media set name is displayed.

Media Pools Locked by single backup job.
Currently one custom media pool is working with one drive. For example if you set up 2 separate media pools for separate jobs – two drives will be used simultaneously.

Status Display / Duration for Continuous Tape Jobs

there is no quick way to see which job is actually running (until the completed percentage increments).

I believe the job duration could be checked fast thru VeeamOne or running the specific PowerShell script.

When the tape drives are free, the timer should start.

Will check on that as your point makes sense to me.

Media Pools are associated with Libraries

all the media pools remained associated with the old library partition.

This is not expected behavior; we will investigate it with QA department.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools

Let’s say you have two libraries one remote another is local using one media pool (containing all the tape media from both libraries) the local would start a job and pick a first media from the pool, which could be media loaded in the remote library so the job would fail. Scenario from the top of my head but it shows why tape media pool should be library oriented.

Once again thank you for your heads up, I will update this thread as soon as I got more information. Cheers!

[danatk]

Hello danatk,
Thank you for the kind words and outstanding valuable feedback! I am glad and proud that you implemented tape functionality in your production!

No problem - it's early days, but we hope it will go well

No way to identify overwritable media
This functionality is under review right now, and hopefully would be improved in the upcoming versions. Meanwhile, you can create media set's name for different jobs based on your retention – and monitor when tape needs to be loaded under media – offline, where barcode/media set name is displayed.

Thanks....all we need is the tape to be flagged as overwritable in an obvious way. e.g. in backupexec the tape label changes to blue in the UI, so it's easy to see, and there is a column showing the overwrite protection status. It would also be good if you could expose the overwrite protection status for a tape via powershell. We could then have a simple script emailing us a list of which tapes are good to retrieve from the offsite vault and put in our scratch tape box.

Media Pools Locked by single backup job.
Currently one custom media pool is working with one drive. For example if you set up 2 separate media pools for separate jobs – two drives will be used simultaneously.

Yeah, that's what we did, e.g. we have two media pools for 'Full' backups, and split our jobs between them. This works OK - the two drives in our library are occupied most of the time. The problem is this uses more tapes in our weekly cycle than we would with one media pool. Also, if all the jobs associated with one media pool finish before the other media pool jobs are done, one of the drive sits idle.

Status Display / Duration for Continuous Tape Jobs I believe the job duration could be checked fast thru VeeamOne or running the specific PowerShell script.

OK we will look at the powershell/veeamone as a temporary fix. However I really think you guys need a 'queued' status for jobs. This is pretty standard in backup software and would mean that triggered jobs that can't start for whatever reason are differentiated from jobs that have started and are either in progress, or are waiting for resources mid-way through a job.

Media Pools are associated with Libraries This is not expected behavior; we will investigate it with QA department.

It's easy to reproduce - simply make your tape library go offline by disconnecting it or similar. The media pools associated with the library are then also offline, and it is not possible to move tapes from these pools to another pool.

Let’s say you have two libraries one remote another is local using one media pool (containing all the tape media from both libraries) the local would start a job and pick a first media from the pool, which could be media loaded in the remote library so the job would fail. Scenario from the top of my head but it shows why tape media pool should be library oriented.

I'm not sure what you mean by a remote library - my understanding is that only locally attached libraries are supported? Or are you thinking of future support?

In which case maybe just make the media pools global to an instance of Veeam B&R, rather than exclusive to a library within that instance?

I hate to refer to backupexec again, it is painful in many ways but the media handling works well. The media pools are not associated with libraries and you can target a job to any device/media pool, move tapes freely between pools etc.

[tsightler]

Yeah, that's what we did, e.g. we have two media pools for 'Full' backups, and split our jobs between them. This works OK - the two drives in our library are occupied most of the time. The problem is this uses more tapes in our weekly cycle than we would with one media pool. Also, if all the jobs associated with one media pool finish before the other media pool jobs are done, one of the drive sits idle.

I'm wondering if you could expound on this portion a little more. I' too would like to see a media pool being able to support multiple drives, however, I'm not really sure how having two pools requires you to use more tapes. If a single pool allowed use of multiple drives you'd still need two jobs going to the same pool which would still use two tapes (and likely only partially use both). Or are you thinking of a more sophisticated case where data from a single job is streamed across multiple drives? Thanks!

[danatk]

I'm wondering if you could expound on this portion a little more. I' too would like to see a media pool being able to support multiple drives, however, I'm not really sure how having two pools requires you to use more tapes. If a single pool allowed use of multiple drives you'd still need two jobs going to the same pool which would still use two tapes (and likely only partially use both). Or are you thinking of a more sophisticated case where data from a single job is streamed across multiple drives? Thanks!

Yeah you might be right...I was thinking that with two pools we would end up with more partially filled tapes, but having thought about it some more we are probably still going to have the same amount at the end of the cycle.

I will keep an eye on it and see how it goes.

I'm not so concerned about number of tapes though, more about keeping the drives spinning and maximising the available backup window.

[veremin]

Assuming that your mediasets are named in default manner, such as “Media set created 8/21/2013 18:49”, “Media set created 9/22/2013 12:33”, etc, you can, probably, use the following script:

Code: Select all

asnp VeeamPssnapin
$Date = Get-Date
$Retention = 30
$Medias = Get-VBRTapeMedium | Sort-Object {$_.FindMediaFamily().name} | Select-Object -Property @{N="MediaSet";E={$_.FindMediaFamily().name}}, @{N="Name";E={$_.name}} ,@{N="Barcode";E={$_.barcode}}, @{N="Creation date";E={$_.FindMediaFamily().name.Substring(18,($_.FindMediaFamily().name.Substring(18).IndexOf(' ')))}}, @{N="Days overdue"; E = {($date - (Get-Date $_.FindMediaFamily().name.Substring(18,($_.FindMediaFamily().name.Substring(18).IndexOf(' '))))).days - $Retention}}
$Medias | where {($date - (Get-Date $_.mediaset.Substring(18,($_.mediaset.Substring(18).IndexOf(' '))))).days -ge $Retention} 

You need to specify number of days as $Retention variable, and script will do the rest, outputting barcodes of tapes that belonged to mediasets with overdue retention period. The script will also output the creation time of mediaset, name of tape, its barcode, and days overdue.

In fact, it’s a draft version, so I don’t mind you polishing it further.

Hope this helps.
Thanks.

[danatk]

Hi - Thanks for the script. We are not naming media sets in the standard way as we found that too many media sets were being created this way - we have 40+ tape jobs and 4 media pools, so new media sets were being created all the time. We had no use for this, so just named the media sets the same as the media pools.

I will review and test the media pool naming to see if the script would be useful for us.

In the meantime if you can implement a UI or powershell feature to expose the overwrite status, that would be great

[danatk]

Yeah you might be right...I was thinking that with two pools we would end up with more partially filled tapes, but having thought about it some more we are probably still going to have the same amount at the end of the cycle.

I will keep an eye on it and see how it goes.

Quick update on this - we completed a full backup run last night, with our backups split across two media pools. When all the jobs finished, we have two partially filled tapes. With one pool I think we would only have one partially filled tape...so one less to send to the vault / free for other backups. Not a big deal

[Dima P.]

danatk,

we have two partially filled tapes

Tape space usage additionally could be controlled thru media set creation option:

• Do not create, continue using the current media set - will use all free space on tape.
• Daily at - will use all the tape free space before new media set is created.
• Create a new media set for every backup session - will use only space needed for the running job.

So that’s not only provide you the scalability of tape media export but controls the free space usage as well.

[veremin]

We are not naming media sets in the standard way as we found that too many media sets were being created this way

Yep, the proposed script will only work in case of Mediaset being named in default manner, such as “Media set created 4/26/2013 19:13”, etc.

The script takes the name of mediaset, or more specifically, portion of it (4/26/2013), then, it converts that portion to the date, and compare the resulting date with the specified “Overdue” ($Retention) period. After that, it lists the name and barcode of medias that belong to mediasets with overdue retention period.

In the meantime if you can implement a UI or powershell feature to expose the overwrite status, that would be great

At the moment I don’t have a chance to test it in our lab, but you can try the following script and see whether it helps you or not:

Code: Select all

Ansp VeeamPSSnaping
$MediaPool = Get-VBRTapeMediaPool -name "Name of Media Pool"
$MediaPool.GetTapeToOverwrite() 

Hope this helps.
Thanks.

[dualdj1]

Media Pools are associated with Libraries

During my test period I had to flash the firmware on my tape library and recreate it's partition presented to Veeam. This was then identified as a new library by Veeam and all the media pools (and media) remained associated with the old (now offline in Veeam) library partition. This was OK for me in testing, as I just trashed the old library in Veeam and created new media pools etc. However, if this was a year or more into a production set-up I would have had a major problem identifying and rebuilding all my media pools.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools. Then we can retire libraries, add new libraries etc without issue.

I can verify that this same issue happened to me as well, after upgrading the firmware on my library. Mine still did come online after the upgrade, but was labeled wrong and had some issues that required me to remove it, and let it redetect, at which point I had to recreate my pools. So maybe an option of some sort to migrate pools to a new library?

[Dima P.]

Jason,

I confirmed that issue after the firmware update may occur, however it is very sporadic and it is recommended to contact support and provide all the troubleshooting info for the further investigation. Thank you.

[tsightler]

Quick update on this - we completed a full backup run last night, with our backups split across two media pools. When all the jobs finished, we have two partially filled tapes. With one pool I think we would only have one partially filled tape...so one less to send to the vault / free for other backups. Not a big deal

Not really since you want to use both drives at the same time thus two partially filled tapes are likely to be the result no matter what. If you want to have a single full tape you'd have to use only one drive which you could do right now with a single job and media pool.

[rgcooper]

Yes, but with a reclamation process (TSM) you could consolidate 2 tapes down 1 after tape backups were complete.

[tsightler]

But what's the point in doing this with Veeam? That would just take even longer. Why not just run all jobs to one tape to start with? Doesn't TSM typically do this when it streams the initial backup directly to tape so that it can get the backup itself done in the fastest possible time? Veeam backs up VMs to disk first, so the disk-to-tape process is already a post-process.

Note that I'm asking this question honestly so that if there's a use case I can actually understand it. I could potentially see this with "File to tape", but it doesn't seem to make any sense to do this with "Backups to tape". With the latter you either want to use multiple drives to get them to tape faster, or you want to use one drive to use less tapes. It doesn't make sense to me to stream the backup files to multiple tapes only to then have to reprocess all that data to a single tape.

[guillermo.lozano]

Hi Community & Veeam Developers,

I would like to request that the standalone drives can write the tapes on a media pool when several tapes have been used and the prior tapes still have space left.

Thanks.
Regards,
Guillermo.-

[veremin]

Hi, Guillermo, could you elaborate on it a little bit? From my perspective, "Don't create, always continue using current media set" option of media pool seems to provide exactly what you're after. With this option, the data will be appended to tape that is currently present, even if this tape has some previous data.

Thanks.

[guillermo.lozano]

Hi Vladimir, i pursued a way around this with support, but finally the conclusion was that it is not supported that Veeam uses a cartridge half way, then use another one, and then come back to the first one (please see Case # 00439919) because it was designed for tape libraries rather than standalone devices.

My request is to get that support for standalone drives.

Thanks.
Regards,
Guillermo.-

[veremin]

So, you’re saying that even with “Do not create, always continue using current media set”, the backup console insists that you insert a new tape, when the job is finished, even though there is a space on a current medium, right?

Thanks.

[guillermo.lozano]

The exact issue is that after a job has finished and the tape has space left, if i use a new tape and after that tape was used and i want to use the first tape with free space left, the job says that the media is invalid.
Support Team told me that this is normal because this was not desinged to use with standalone drives, and this kind operation is not suported, so that's why i'm asking the Feature.
thanks.

[veremin]

In case of standalone drives, when you insert new cassette, new mediaset is automatically created.

The only case when the current mediaset continues is when one cassette isn’t enough to host the corresponding data and you insert a new clean tape. If you insert cassette that has some backup data already written to it, then, you will see the said error “Tape is invalid”.

So, you’re right – currently, such scenario isn’t supported. The only option you have is not to append data to tape with previous information, but rather automatically overwrite it, using retention setting of a given media pool.

Anyway, thanks for the feedback, I’ve already passed this information to our OA team; much appreciated.

[guillermo.lozano]

Thank You Vladimir.
Best Regards,
Guillermo.-

[veremin]

You're welcome. Feel free to ask, if any additional help or clarification is needed. Thanks.

[danatk]

Code: Select all

Ansp VeeamPSSnaping
$MediaPool = Get-VBRTapeMediaPool -name "Name of Media Pool"
$MediaPool.GetTapeToOverwrite() 

Hope this helps.
Thanks.

Hi - I tried this out and it doesn't seem to work, unless I'm going wrong

I ran the command and it showed only 1 tape free for overwrite, which didn't seem correct, so in order to test I set the overwrite period on the media pool to 'Do not protect data' and ran the command again. It still only showed one tape available for overwrite, where I would have expected all tapes to now show as overwritable.

This is the main feature that I'm missing from Veeam for tapes...the ability to see which tapes are available for overwrite. It should be immediately obvious in the UI.

thanks,
Dan.

[veremin]

Hmm, then, may be the following script will be helpful. This script gets all tapes in a given media poll that can be overwritten and shows their names:

Code: Select all

asnp VeeamPSSnapin
$MediaPool = Get-VBRTapeMediaPool -name "Name of the media pool"
foreach ($Medium in ($MediaPool.GetTapeMediums() | where {$_.IsOverwriteProtectionPeriodOver() -eq $True}))
{
$Medium.name
}

Hope this helps.
Thanks.

[danatk]

yes that seems to work, thanks!

[Toschwil]

Hi there,

First I want to apologize for my english, I'm from Germany.
I now also use the R2 version, because I also had the error that prevents tapes were overwritten each backup job. I wish that Veeam continues to expand the tape support, so that customers actually could do without another backup software for tapes.
Currently, there are for me several limitations that prevent this effect.
First, here reminds me of the fact a that tape jobs run 8 days, because they are waiting for a matching Tape and meanwhile block all other tape jobs. Veeam V7 has not built in any warnings that a tape job will run as long . There is no switch where i could specify that after 5 minutes of " waiting for a suitable tape" the active job is canceled.
Why can veeam not ( like many other tape software also ) just the (Tape) media pool divided into a working- and a saveset, the administrator at a glance could see if everything is running properly and quickly check if for the next Tapejob also tapes are available.
Why is it not possible to have a "File-tape job" to append to a tape job, so another directory ends up with more on the same tape?
But now I have bitched enough, there are plans one or the other function promptly implement?

[Dima P.]

Hello Thomas,
Thank you for your feedback!

First, here reminds me of the fact a that tape jobs run 8 days, because they are waiting for a matching Tape and meanwhile block all other tape jobs. Veeam V7 has not built in any warnings that a tape job will run as long . There is no switch where i could specify that after 5 minutes of " waiting for a suitable tape" the active job is canceled.

In the next releases we are planning to provide the granular notification for backup to tape jobs, so in case you backup to tape is hanging for some reason - tape operator would get an email notification. As for the "timeout" feature for tape jobs will be discussed with our development team.

Why can veeam not ( like many other tape software also ) just the (Tape) media pool divided into a working- and a saveset, the administrator at a glance could see if everything is running properly

By design tape library “owns” media pool, and media pool “owns” tape media. You can point all your tape jobs to one media pool, or use separate media pools to organize different retentions. Additionally you can have one backup to tape job including all your backups, or backup to tape for each backup.

quickly check if for the next Tapejob also tapes are available.

There is no need to do that if you use pick up free tape media from free media pool in your media pool options (step 2 Media Pool > Tapes)

Why is it not possible to have a "File-tape job" to append to a tape job, so another directory ends up with more on the same tape?

Are you talking about the ability to share one tape with free space between multiple tape jobs?

I wish that Veeam continues to expand the tape support

Yes, we are going to do that.

[veremin]

Additionally, you can put into use Veeam ONE that allows you to keep track of the job duration and send you notification in case specified interval is exceeded. Thanks.

[Brett76]

Hi all,

I have recently installed Veeam in our environment after many issues with our previous backup product. I am very impressed with the product and really enjoy using it.

The only issue have have come up against is the tape media pools being library orientated and would also like to request that media pools be agnostic to the library.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools

Let’s say you have two libraries one remote another is local using one media pool (containing all the tape media from both libraries) the local would start a job and pick a first media from the pool, which could be media loaded in the remote library so the job would fail. Scenario from the top of my head but it shows why tape media pool should be library oriented.

In my situation, I have 2 x HP autoloaders attached to my backup server. When I setup a job to go to tape, I do not really care which autoloaded it is backed up to, I would rather it go to the first available drive. I dont want to have to worry about balancing jobs between the 2 autoloaders. I would also get redundancy by having the media pools independant of the libraries. Of couse the issue of locking the media pool when a backup is in progress would also need to be resolved.

Thanks
Brett