Tape Support - Feedback / Feature Req

Everything about backing up to tape

Tape Support - Feedback / Feature Req

Veeam Logoby danatk » Thu Oct 24, 2013 9:10 am

Hi Guys - We love the new tape support in Veeam - we have been testing for 3 weeks and have now gone into production with a working implementation. We really think it's great for a first version, but there are of course a few things we'd like to see improved :D

No way to identify overwritable media

We can set an overwrite protection period on media sets, but there's no way to see whether a tape is ready for overwrite. Therefore we have no way to easily tell which tapes to retrieve from our offsite store to be overwritten.

Media Pools Locked by single backup job.

Why does the entire media pool get locked by a single job? This means if we have multiple tape drives that we want to run simultaneous jobs, we need multiple media pools for different jobs, or only one drive is used.

Status Display / Duration for Continuous Tape Jobs

It would be nice if jobs had a 'paused' or 'waiting' status when waiting for backup resources. At present we can have multiple jobs running, many of which are waiting for resources, and there is no quick way to see which job is actually running (until the completed percentage increments).

Also, if jobs start and are waiting for resources this increments the duration timer which means we get confusing job durations. For example, we have many continuous tape jobs which start when new backup files from VM jobs appear. These jobs often start when the tape drives are busy with other jobs - at this point they should queue, but not start their timer. When the tape drives are free, the timer should start. This would mean the job duration accurately showed the duration of the job, rather than the time it was queued + duration.

Media Pools are associated with Libraries

During my test period I had to flash the firmware on my tape library and recreate it's partition presented to Veeam. This was then identified as a new library by Veeam and all the media pools (and media) remained associated with the old (now offline in Veeam) library partition. This was OK for me in testing, as I just trashed the old library in Veeam and created new media pools etc. However, if this was a year or more into a production set-up I would have had a major problem identifying and rebuilding all my media pools.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools. Then we can retire libraries, add new libraries etc without issue.
danatk
Influencer
 
Posts: 19
Liked: 1 time
Joined: Thu Oct 24, 2013 8:58 am

Re: Tape Support - Feedback / Feature Req

Veeam Logoby Dima P. » Thu Oct 24, 2013 2:19 pm

Hello danatk,
Thank you for the kind words and outstanding valuable feedback! I am glad and proud that you implemented tape functionality in your production!

No way to identify overwritable media
This functionality is under review right now, and hopefully would be improved in the upcoming versions. Meanwhile, you can create media set's name for different jobs based on your retention – and monitor when tape needs to be loaded under media – offline, where barcode/media set name is displayed.

Media Pools Locked by single backup job.
Currently one custom media pool is working with one drive. For example if you set up 2 separate media pools for separate jobs – two drives will be used simultaneously.

Status Display / Duration for Continuous Tape Jobs
there is no quick way to see which job is actually running (until the completed percentage increments).

I believe the job duration could be checked fast thru VeeamOne or running the specific PowerShell script.

When the tape drives are free, the timer should start.

Will check on that as your point makes sense to me.

Media Pools are associated with Libraries
all the media pools remained associated with the old library partition.

This is not expected behavior; we will investigate it with QA department.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools

Let’s say you have two libraries one remote another is local using one media pool (containing all the tape media from both libraries) the local would start a job and pick a first media from the pool, which could be media loaded in the remote library so the job would fail. Scenario from the top of my head but it shows why tape media pool should be library oriented.

Once again thank you for your heads up, I will update this thread as soon as I got more information. Cheers!
Dima P.
Veeam Software
 
Posts: 6249
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Tape Support - Feedback / Feature Req

Veeam Logoby danatk » Thu Oct 24, 2013 4:57 pm

d.popov wrote:Hello danatk,
Thank you for the kind words and outstanding valuable feedback! I am glad and proud that you implemented tape functionality in your production!


No problem - it's early days, but we hope it will go well :)


No way to identify overwritable media
This functionality is under review right now, and hopefully would be improved in the upcoming versions. Meanwhile, you can create media set's name for different jobs based on your retention – and monitor when tape needs to be loaded under media – offline, where barcode/media set name is displayed.


Thanks....all we need is the tape to be flagged as overwritable in an obvious way. e.g. in backupexec the tape label changes to blue in the UI, so it's easy to see, and there is a column showing the overwrite protection status. It would also be good if you could expose the overwrite protection status for a tape via powershell. We could then have a simple script emailing us a list of which tapes are good to retrieve from the offsite vault and put in our scratch tape box.

Media Pools Locked by single backup job.
Currently one custom media pool is working with one drive. For example if you set up 2 separate media pools for separate jobs – two drives will be used simultaneously.


Yeah, that's what we did, e.g. we have two media pools for 'Full' backups, and split our jobs between them. This works OK - the two drives in our library are occupied most of the time. The problem is this uses more tapes in our weekly cycle than we would with one media pool. Also, if all the jobs associated with one media pool finish before the other media pool jobs are done, one of the drive sits idle.

Status Display / Duration for Continuous Tape Jobs
there is no quick way to see which job is actually running (until the completed percentage increments).

I believe the job duration could be checked fast thru VeeamOne or running the specific PowerShell script.


OK we will look at the powershell/veeamone as a temporary fix. However I really think you guys need a 'queued' status for jobs. This is pretty standard in backup software and would mean that triggered jobs that can't start for whatever reason are differentiated from jobs that have started and are either in progress, or are waiting for resources mid-way through a job.

Media Pools are associated with Libraries
all the media pools remained associated with the old library partition.

This is not expected behavior; we will investigate it with QA department.


It's easy to reproduce - simply make your tape library go offline by disconnecting it or similar. The media pools associated with the library are then also offline, and it is not possible to move tapes from these pools to another pool.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools

Let’s say you have two libraries one remote another is local using one media pool (containing all the tape media from both libraries) the local would start a job and pick a first media from the pool, which could be media loaded in the remote library so the job would fail. Scenario from the top of my head but it shows why tape media pool should be library oriented.


I'm not sure what you mean by a remote library - my understanding is that only locally attached libraries are supported? Or are you thinking of future support?

In which case maybe just make the media pools global to an instance of Veeam B&R, rather than exclusive to a library within that instance?

I hate to refer to backupexec again, it is painful in many ways but the media handling works well. The media pools are not associated with libraries and you can target a job to any device/media pool, move tapes freely between pools etc.
danatk
Influencer
 
Posts: 19
Liked: 1 time
Joined: Thu Oct 24, 2013 8:58 am

Re: Tape Support - Feedback / Feature Req

Veeam Logoby tsightler » Thu Oct 24, 2013 5:05 pm

danatk wrote:Yeah, that's what we did, e.g. we have two media pools for 'Full' backups, and split our jobs between them. This works OK - the two drives in our library are occupied most of the time. The problem is this uses more tapes in our weekly cycle than we would with one media pool. Also, if all the jobs associated with one media pool finish before the other media pool jobs are done, one of the drive sits idle.


I'm wondering if you could expound on this portion a little more. I' too would like to see a media pool being able to support multiple drives, however, I'm not really sure how having two pools requires you to use more tapes. If a single pool allowed use of multiple drives you'd still need two jobs going to the same pool which would still use two tapes (and likely only partially use both). Or are you thinking of a more sophisticated case where data from a single job is streamed across multiple drives? Thanks!
tsightler
Veeam Software
 
Posts: 4769
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Tape Support - Feedback / Feature Req

Veeam Logoby danatk » Thu Oct 24, 2013 6:57 pm

tsightler wrote:
danatk wrote:Yeah, that's what we did, e.g. we have two media pools for 'Full' backups, and split our jobs between them. This works OK - the two drives in our library are occupied most of the time. The problem is this uses more tapes in our weekly cycle than we would with one media pool. Also, if all the jobs associated with one media pool finish before the other media pool jobs are done, one of the drive sits idle.


I'm wondering if you could expound on this portion a little more. I' too would like to see a media pool being able to support multiple drives, however, I'm not really sure how having two pools requires you to use more tapes. If a single pool allowed use of multiple drives you'd still need two jobs going to the same pool which would still use two tapes (and likely only partially use both). Or are you thinking of a more sophisticated case where data from a single job is streamed across multiple drives? Thanks!


Yeah you might be right...I was thinking that with two pools we would end up with more partially filled tapes, but having thought about it some more we are probably still going to have the same amount at the end of the cycle.

I will keep an eye on it and see how it goes.

I'm not so concerned about number of tapes though, more about keeping the drives spinning and maximising the available backup window.
danatk
Influencer
 
Posts: 19
Liked: 1 time
Joined: Thu Oct 24, 2013 8:58 am

Re: Tape Support - Feedback / Feature Req

Veeam Logoby v.Eremin » Fri Oct 25, 2013 11:07 am

Assuming that your mediasets are named in default manner, such as “Media set created 8/21/2013 18:49”, “Media set created 9/22/2013 12:33”, etc, you can, probably, use the following script:

Code: Select all
asnp VeeamPssnapin
$Date = Get-Date
$Retention = 30
$Medias = Get-VBRTapeMedium | Sort-Object {$_.FindMediaFamily().name} | Select-Object -Property @{N="MediaSet";E={$_.FindMediaFamily().name}}, @{N="Name";E={$_.name}} ,@{N="Barcode";E={$_.barcode}}, @{N="Creation date";E={$_.FindMediaFamily().name.Substring(18,($_.FindMediaFamily().name.Substring(18).IndexOf(' ')))}}, @{N="Days overdue"; E = {($date - (Get-Date $_.FindMediaFamily().name.Substring(18,($_.FindMediaFamily().name.Substring(18).IndexOf(' '))))).days - $Retention}}
$Medias | where {($date - (Get-Date $_.mediaset.Substring(18,($_.mediaset.Substring(18).IndexOf(' '))))).days -ge $Retention}


You need to specify number of days as $Retention variable, and script will do the rest, outputting barcodes of tapes that belonged to mediasets with overdue retention period. The script will also output the creation time of mediaset, name of tape, its barcode, and days overdue.

In fact, it’s a draft version, so I don’t mind you polishing it further.

Hope this helps.
Thanks.
v.Eremin
Veeam Software
 
Posts: 13279
Liked: 969 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Tape Support - Feedback / Feature Req

Veeam Logoby danatk » Fri Oct 25, 2013 7:33 pm

Hi - Thanks for the script. We are not naming media sets in the standard way as we found that too many media sets were being created this way - we have 40+ tape jobs and 4 media pools, so new media sets were being created all the time. We had no use for this, so just named the media sets the same as the media pools.

I will review and test the media pool naming to see if the script would be useful for us.

In the meantime if you can implement a UI or powershell feature to expose the overwrite status, that would be great :D
danatk
Influencer
 
Posts: 19
Liked: 1 time
Joined: Thu Oct 24, 2013 8:58 am

Re: Tape Support - Feedback / Feature Req

Veeam Logoby danatk » Sat Oct 26, 2013 10:10 am

Yeah you might be right...I was thinking that with two pools we would end up with more partially filled tapes, but having thought about it some more we are probably still going to have the same amount at the end of the cycle.

I will keep an eye on it and see how it goes.


Quick update on this - we completed a full backup run last night, with our backups split across two media pools. When all the jobs finished, we have two partially filled tapes. With one pool I think we would only have one partially filled tape...so one less to send to the vault / free for other backups. Not a big deal :D
danatk
Influencer
 
Posts: 19
Liked: 1 time
Joined: Thu Oct 24, 2013 8:58 am

Re: Tape Support - Feedback / Feature Req

Veeam Logoby Dima P. » Mon Oct 28, 2013 9:46 am

danatk,

we have two partially filled tapes

Tape space usage additionally could be controlled thru media set creation option:

• Do not create, continue using the current media set - will use all free space on tape.
• Daily at - will use all the tape free space before new media set is created.
• Create a new media set for every backup session - will use only space needed for the running job.

So that’s not only provide you the scalability of tape media export but controls the free space usage as well.
Dima P.
Veeam Software
 
Posts: 6249
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Tape Support - Feedback / Feature Req

Veeam Logoby v.Eremin » Mon Oct 28, 2013 10:21 am

We are not naming media sets in the standard way as we found that too many media sets were being created this way

Yep, the proposed script will only work in case of Mediaset being named in default manner, such as “Media set created 4/26/2013 19:13”, etc.

The script takes the name of mediaset, or more specifically, portion of it (4/26/2013), then, it converts that portion to the date, and compare the resulting date with the specified “Overdue” ($Retention) period. After that, it lists the name and barcode of medias that belong to mediasets with overdue retention period.

In the meantime if you can implement a UI or powershell feature to expose the overwrite status, that would be great :D

At the moment I don’t have a chance to test it in our lab, but you can try the following script and see whether it helps you or not:

Code: Select all
Ansp VeeamPSSnaping
$MediaPool = Get-VBRTapeMediaPool -name "Name of Media Pool"
$MediaPool.GetTapeToOverwrite()


Hope this helps.
Thanks.
v.Eremin
Veeam Software
 
Posts: 13279
Liked: 969 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Tape Support - Feedback / Feature Req

Veeam Logoby dualdj1 » Mon Oct 28, 2013 7:57 pm

danatk wrote:Media Pools are associated with Libraries

During my test period I had to flash the firmware on my tape library and recreate it's partition presented to Veeam. This was then identified as a new library by Veeam and all the media pools (and media) remained associated with the old (now offline in Veeam) library partition. This was OK for me in testing, as I just trashed the old library in Veeam and created new media pools etc. However, if this was a year or more into a production set-up I would have had a major problem identifying and rebuilding all my media pools.

The media pools should be agnostic to the library...i.e. we should be able to have multiple libraries that use common media pools. Then we can retire libraries, add new libraries etc without issue.


I can verify that this same issue happened to me as well, after upgrading the firmware on my library. Mine still did come online after the upgrade, but was labeled wrong and had some issues that required me to remove it, and let it redetect, at which point I had to recreate my pools. So maybe an option of some sort to migrate pools to a new library?
dualdj1
Enthusiast
 
Posts: 43
Liked: 4 times
Joined: Tue Feb 05, 2013 6:56 pm
Full Name: Jason K. Brandt

Re: Tape Support - Feedback / Feature Req

Veeam Logoby Dima P. » Mon Oct 28, 2013 11:06 pm

Jason,

I confirmed that issue after the firmware update may occur, however it is very sporadic and it is recommended to contact support and provide all the troubleshooting info for the further investigation. Thank you.
Dima P.
Veeam Software
 
Posts: 6249
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Tape Support - Feedback / Feature Req

Veeam Logoby tsightler » Tue Oct 29, 2013 11:58 am

danatk wrote:Quick update on this - we completed a full backup run last night, with our backups split across two media pools. When all the jobs finished, we have two partially filled tapes. With one pool I think we would only have one partially filled tape...so one less to send to the vault / free for other backups. Not a big deal :D


Not really since you want to use both drives at the same time thus two partially filled tapes are likely to be the result no matter what. If you want to have a single full tape you'd have to use only one drive which you could do right now with a single job and media pool.
tsightler
Veeam Software
 
Posts: 4769
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Tape Support - Feedback / Feature Req

Veeam Logoby rgcooper » Tue Oct 29, 2013 1:34 pm

Yes, but with a reclamation process (TSM) you could consolidate 2 tapes down 1 after tape backups were complete.
rgcooper
Novice
 
Posts: 4
Liked: never
Joined: Fri Apr 26, 2013 8:13 pm
Full Name: Robert Cooper

Re: Tape Support - Feedback / Feature Req

Veeam Logoby tsightler » Tue Oct 29, 2013 3:11 pm

But what's the point in doing this with Veeam? That would just take even longer. Why not just run all jobs to one tape to start with? Doesn't TSM typically do this when it streams the initial backup directly to tape so that it can get the backup itself done in the fastest possible time? Veeam backs up VMs to disk first, so the disk-to-tape process is already a post-process.

Note that I'm asking this question honestly so that if there's a use case I can actually understand it. I could potentially see this with "File to tape", but it doesn't seem to make any sense to do this with "Backups to tape". With the latter you either want to use multiple drives to get them to tape faster, or you want to use one drive to use less tapes. It doesn't make sense to me to stream the backup files to multiple tapes only to then have to reprocess all that data to a single tape.
tsightler
Veeam Software
 
Posts: 4769
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 8 guests