Tape is full ???

[marksmithuk]

I dont suppose anyone else has experienced an issue with a backup to tape saying the tape is full even though there should be plenty of space left on it?

I have a Veeam backup job that is 1.1TB in size, I am using a LTO5 drive and tape with a native capacity of 1.5TB, so I should be fine right? When I inventory the drive it says the tape is blank and has the full capacity avaliable, but when I do the backup it gets to about 1TB and says tape is full.

Anyone else ever had anything like this or have any ideas why it might be happening?

I have tried with hardware compression on and off. We are using an IBM SCSI backup drive and I have tried our 2nd drive with new tapes and it is doing exactly the same. Any help would be aprechiated.

[Dima P.]

Hello Mark,
I guess this requires deep investigation as it is not expected behavior. Could I ask you to open a support case and submit the following information:

- Complete backup to tape job details
- Exact model of the tape drives you used
- Veeam logs and Veeam service logs

Please also update this post with the original case number so we could take part in the investigation procedure.

I will await for your response,
Thank you

[marksmithuk]

Thanks, the case is ref# 00432190

[yizhar]

Hi.

Just to make sure - did you verify that the media itself (and not only the drive) is LTO5 and not LTO4?

Can you try to run a "file backup to tape" instead of VM backup and compare the results?

Can you try with hardware compression both turned on or off?

Yizhar

[marksmithuk]

Thanks for the ideas, yep all are LTO5, tried old and new tapes as well just to make sure it wasnt a dodgy tape or batch. I have tried running a file backup of just the vbk and also a job backup to tape, both with simular results. However the file one did seem to back up slightly more, but not much.

I have also tried with compression on and off with both file and job style backups.

[Dima P.]

Mark thank you for the update!

I just wanted to follow up and say that we are investigating the issue and will update this post and your case as soon as we find out what causing this strange behavior.
Thank you for your patience.

[stuartmacgreen]

Any update on this. Or root cause? Seeing some symptoms in Veeam 8.

[veremin]

Hi, Stuart, do you have ticket open on that behaviour? Typically, you might face this issue due tape drive misnotifying us that current tape is full, when in reality it is not. Thanks.

[stuartmacgreen]

SR 00685458 is already open for another problem but I think they are linked now as I am seeing more information about media.
This was to do with block size on tapes.

[veremin]

Yep, I can imagine that some problems with drive misinforming backup server about tape fulness might be related to the issue with wrong identification of tape block size. Keep working with support team on addressing that, and thank you for providing additional information.

[stuartmacgreen]

I have updated firmware on my drives today and already see something different in the logs.

FYI: LTO 5 Ultrium 3280 FC drives from HP in a MSL8096

Upgraded MSL8096 from 1120 to 1130 firmware
Upgraded 3280 drives from code I65W to I6GW

the release notes show some interesting fixes and implementations.

Improved tape space usage for a few corner case patterns.

[veremin]

As mentioned, we rely on tape device to get up-to-date information regarding tape free space. If for some reason (firmware, deterioration) the device starts to misbehave, we will be automatically affected. So, does the problem persist on a new firmware? Thanks.

[stuartmacgreen]

Problem still persists on newer firmware. The support advice for the problems I face with block size is to perform Long Erase on all medium and go from there as in nearly all case this fixes that issue.

For anyone else here is the error message I got.

Failed to prepare to start new tape session: StartSessionTape failed
StartSessionTape failed
When accessing a new tape of a multivolume partition, the current block size is incorrect.
Tape read error

Apparently there is changed logic in v8. Now block size is considered in order to have less issues with restore from tapes.

So at 3hrs per LTO-5 tape it is a long process. At least I got 4 drives and only 80 tapes to operate on over time as they expire.