Failed to eject some tapes

[jimbliss]

After updating to 9.5, we are getting a "Failed to eject some tapes" on some of our tape jobs. After watching it, the tape is ejecting, and the robot is moving it back to its slot correctly. It looks like a timing issue, in that the library robot is taking longer that the time to wait Veeam 9.5 allows. We had no issues with v9.0.

My question, is there such a setting? And can it be changed?

The library is an HPE MSL8096 G3 with LTO6 FC drives. All firmware and drivers are at the latest level. OS is MS Windows Server 2012 R2 running on a BL460 G8.

Thanks,

Jim

[Mike Resseler]

Hi Jim,

As far as I know, there is no specific setting for this. I also can't find a specific registry key for it so I would advise you to log a support case so our support engineers can look at this and try to solve it.

Post the case ID here and the result after that also to help fellow community members

Thanks!

Mike

[francs]

Case #02002409
I'm having the same issue on an IBM TS3200 since upgrading to 9.5

Code: Select all

I found the following warning in the job logfiles.
[09.12.2016 16:02:01] <01> Warning  [CTapeMediumUnloader] Skipping tape media SAQ175L5 eject because lock failed
[09.12.2016 16:02:01] <01> Info     Unlocking tape medium 'SAQ179L5'
[09.12.2016 16:02:01] <01> Info         Tape medium 'SAQ179L5' is successfully unlocked
[09.12.2016 16:02:01] <01> Info     Unlocking device '\\.\Tape0'
[09.12.2016 16:02:01] <01> Info         Device '\\.\Tape0' successfully unlocked
[09.12.2016 16:02:01] <01> Info     [TapeDriveLock] Drive 'Drive 1 (Server: BARNEY, Library: IBM 3573-TL C.50, Drive ID: Tape0)' was unlocked
[09.12.2016 16:02:01] <01> Warning  Failed to eject some tapes

Comparing this to the logs before the upgrade it would seem that it would wait for the unlock to complete first and then do an unload.
This doesn't seem to be the case with v9.5, where the unlock and unload seems to happen at the same time.

[Mike Resseler]

Hi Francs,

Thanks for posting here also. Keep us informed when support gives you more information

Thanks
Mike

[veremin]

There is hot fix for this issue. You may refer to ticket number 01982635 in order to get it. Thanks.

[jimbliss]

Hot fix did the job. Thanks.

[francs]

I can confirm. The hotfix solved the problem.

[veremin]

Thank you, guys, for confirming that hot fix has solved the problem for you. However, it would not harm to check occasionally whether there are no tapes stuck in drives. Thanks.

[kmbelt]

I am having the Exact same problem since upgrading. How can I obtain the hotfix?

[lyapkost]

Hi, Kevin! You need to open a support case and refer to case #01982635 to obtain a hotfix.

By the way. There will be a special registry key DisableEjectWarning available after upgrading to upd1. If it is set to 1, no more warnigs should appear.

[edvschriesheim]

But why should i disable the warning?

[lyapkost]

But why should i disable the warning?

Hi! You don't have to disable it, warnings are turned on by default as it was before. But if for any reason one doesn't want to receive these warnings he has an option to disable them.

[danc]

Veeam Support - Case # 02066873
I am having the same issue since upgrading to 9.5 Update 1:

Tape library: Physical - HP MSL G3 Series 8.70
Used tapes: HOE862L5 [HOE862L5]
TapeDrive alert: The tape drive is having problems reading data. No data has been lost, but there has been a reduction in the performance of the tape..
Failed to eject some tapes
Skipping tape media HOE862L5 eject because lock failed


Is the hotfix available for download or do I need to wait for Veeam support to provide the hotfix via the Case logged?

Thanks
Dan

[Mike Resseler]

Dan,

Did you refer to hotfix (ticket number) 01982635 in your support case? I actually have my doubts this is the same problem to be honest so yes Support will take it further from here and if the hotfix is the solution they will give it to you. But as I said, I have a feeling that there is something else

[dgreer]

I'm going to throw in a "Me too" on this.

We've received four "Warnings" from VB&R in last 40(ish) days: "Failed to eject some tapes. Skipping tape media [media name] eject because lock failed." The backups, themselves, appear to be successful, so no apparent worries there. It appears Veeam is just failing to eject the tapes.

We updated VB&R from v9.0 to v9.5.0.711 on 11/29/16. It performed error free for 49 days, until first occurrence of warning on 01/16/17, and then subsequent warnings on 02/03/17, 02/20/17, and 02/24/17. I googled the warning, and it led me to this forum post.

I don't know if this "hotfix (ticket number) 01982635" is the magic bullet, or if it's included in "9.5 Update 1", but we have not yet installed either piece. I have opened a support case (including reference to aforementioned "hot fix") to request further guidance before making any changes (and provided both screenshots & logs), but any assistance you guys (from Veeam) can provide will be appreciated.

Thanks,
-DG

[dgreer]

Hello, Folks;

Quick follow-up: I received the following response from Veeam Support, this morning:

This is a new warning that was added in version 9. Unfortunately when mutliple jobs are set to use the same media pool of tapes they can give false warnings as you are getting. Once job completes the use of a tape and another uses it before the previous job ejects it, and the first job will give the warning. We have provided a registry key to disable this warning for customers that have this occur.

Please add the registry key below to your Veeam server to disable the unneeded warnings.

Host: Veeam B&R Server
Path: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication
Value: DisableEjectWarning
Type: DWORD
Value : 1

This raises an obvious follow-up question (similar to that asked by edvschriesheim and responded to by lyapkost, both on 1/31/17):

Won't adding/altering this registry key also prevent the warning from being issued in the event of a "real" eject failure?

Perhaps my logic is flawed, but I'm inclined to think it preferable (albeit, granted, annoying) to have to look into an occasional false warning, than to receive no warning at all in the event of a "real" eject failure - especially if a "real" eject failure could prevent or otherwise adversely affect successful retrieval of backed up data, down the road.

If I'm missing something, here, please, by all means, let me know.


Thanks,
-DG

[lyapkost]

Hi, Don!

What do you mean by 'real eject failure'? Imagine you have 2 tape jobs to the same media pool with 'always continue current media set' option enabled. The first job has 'eject' option enabled, the second doesn't. When the first job finishes, the second one takes the last written tape and continues writing. The tape will not be ejected after the second job finishes. Is it a 'real eject failure', a misconfig or what? In my opinion the real failure is when a tape stucks in drive and in that case you will receive 'red' alerts from the library and tape jobs so this registry key won't prevent you from noticing such hardware failures.

Another example is when 'export' option is enabled as well and exported tapes are periodically moved to another location. In that case I suggest not to disable warnings to be sure that all tapes with the backup are successfully ejected and exported (actually the best practice is to have a separate media pool for such backups).

[dgreer]

Hi Konstantin,

Thanks for the response, I appreciate it. I think we're on the same page, regarding a "real" failure, and you raised an interesting point, regarding a misconfig. Our scenario is slightly different than the one you described (we have/use only one "Backup to Tape" job), but there are similarities:

We run four "Backup Jobs" (to file) that back up four servers to the Default Backup Repository. The four jobs run sequentially and, then (upon completion of the fourth "Backup Job"), the "Backup to Tape" job runs, backing up the Default Backup Repository to tape. As you mentioned, though, "always continue..." and "eject media..." are selected ("export" is not). This scenario runs daily (Sun-Fri = incremental, Sat = full), and tapes are also changed daily (per media rotation strategy).

Could be I'm missing "the bigger picture" but, due to the sporadic occurrence of this issue on our end, I'm inclined to believe we may be running into either a timing issue between Veeam and the Tape Library (similar to that mentioned by the OP), or else a tape is getting physically stuck (ie. "real" failure). Unfortunately, I'm 2,000 miles away from the unit, so I'm unable to personally observe the unit's behavior or otherwise verify this theory, but I'm working with the device operator to try to determine that.

Regardless, your response and the one I received in my official support case are virtually identical, and you've both answered my question as to whether or not disabling the registry key will prevent a "red alert" in the event of a true hardware failure (that answer being "no."). As for how we're going to proceed from here, while we certainly appreciate having the option to disable the warning, we've decided to leave the registry key "as-is" (warnings enabled). When it comes to backups, especially with being new to Veeam, I'd prefer to err on the side of caution.

Thanks again for your response and assistance, Konstantin!


Kind regards,
-DG


Support case: 02084013

[GraPghaST]

Setup is a virtual Veeam B&R 9.5.0.823 on Windows server 2012 R2, with a virtual Veeam Tape Appliance on Server 2012 R2, which in-guest connects to a Dell TL4000 through a SAS-iSCSI bridge. Backups, inventory, library operations all work. I only have a warning after running a tape job, saying:

[04.04.2017 18:53:38] <01> Warning [CTapeMediumUnloader] Skipping tape media <media label> eject because lock failed
[04.04.2017 18:53:38] <01> Warning Failed to eject some tapes

I found the same issue in tape-f29/failed-to-eject-some-tapes-t39528.html , in which a hotfix is put available. SC #01982635. Only I am running an NFR key, so no support ID to create my own support case. Is there another way to obtain the hotfix mentioned?

Thanks!

[veremin]

Hi, Robbert,

If you don't want to receive these warning, it stands to reason to install Update 1 and implement the mentioned regkey.

Thanks.

[GraPghaST]

Hi v.Eremin. Thanks for your actions! Though I am not looking for a way to hide the warning, I'd like the tape to get ejected from the drive and stored in a library slot, as the command should do. Now the tape is still in the tape drive, so the command is not executed correctly. Is there a solution for this, too?

[veremin]

Ah, so you're not talking about false positive warnings, but rather about red errors generated due to tape media being stuck in tape library.

Kindly, reach our support team for further investigation. According to our support policy, people with NFS license are eligible for support.

Thanks.

[chalkynz]

This still happens on 9.5 U4b, but the 'failed to eject' message is information only, it is not a warning nor error. This happens every day for me: Two jobs, targeting different media pools, both set to eject tape at end. Job 1 runs first, completes, RELEASES THE TAPE DRIVE, job 2 jumps in, claims the tape drive (I believe this is intended design, in case job 2 can append to tape already in drive), but job 2 wants a different media pool, so job 2 does an eject, and a load, and starts writing to tape. Job 1 cannot access the tape drive, so cannot do an eject, gives up after 3 minutes.

Does Veeam log differently if it's a 'true' eject failure (as opposed to a job hand-over eject failure, like above)? I don't know, I've never had one.