Immutability of tape.

[slackuser]

I've been researching tape immutability for a bit and it seems like the ability for tape to be WORM is really rooted in the firmware of the drive refusing to write over the tape as there is nothing inherently WORM about magnetic media. This seems to indicate that the attack vector would be the drive firmware. Anyone else thinking about this? Anyone know if the microcode on the drive that enforces worm is updated with drive firmware?

Here are some research papers I found on the subject:

https://citeseerx.ist.psu.edu/viewdoc/d ... 1&type=pdf
https://webpages.uncc.edu/yonwang/papers/worm.pdf
https://citeseerx.ist.psu.edu/viewdoc/d ... 1&type=pdf

Looks like the LTO worm stuff is technically E-WORM which doesn't offer the same level of security assurance as something that has WORM rooted in the physics of the media.

[Dima P.]

Hello Matthew,

The beauty of the tape - you can export it offline

[soncscy]

Hey Matthew,

I read the papers, and I'm not convinced they have much of a threat vector on E-WORM like they suggest. Only the last article hints at a potential to attack at the driver/firmware level without really demonstrating a practical attack, and frankly speaking, if such an attacker vector is possible on your WORM tapes, it's missing the point of tapes.

As Dima wrote, just move the tape offline.

You don't use tape because of a bullet proof indestructible firmware foundation (though it is quite good at this point in Tape's life); you use tape because it's a ton of storage that you can take out easily. LTO6 and later tapes are fine and safe from environmental concerns just long enough for you to get them in a vault somewhere (your own or managed).

The papers are...well, amusing, but the attacks they propose are along the lines of "worrying about the wallpaper while your castle is burning" in my opinion. To get the level of access necessary to pooch WORM tapes, an attacker would already need to have enough permissions/access to wreck everything else.

Where there is a threat vector is the malicious insider either at the company or the vault you use, and I do agree on this as a huge problem; but once physical access is a factor, software solutions for protection go right out the window and you cannot guarantee anything, especially since with tape a successful attack can be as simple as just take a hammer + some scissors to the tape.

So, amusing articles, but I think they mostly point out irrelevant attack vectors.

[RobTurk]

The claim in this paper is a bit of a stretch.

"Magnetic tape WORM involves a combination of tape storage and firmware based protection techniques. Commonly used tape is not usually write-protected, so this requires specialized tape drives with embedded write protection mechanism"

This is not correct. WORM functionality is part of the LTO standard since LTO-3. All cartridges contain a RFID chip with metadata, and one of the settings is WORM or not. If the cartridge is marked WORM, the drive will refuse to overwrite it. So rather than requiring specialized drives to use WORM, you'd have to have specialized drives with hacked firmware to overwrite a WORM tape. Quite a challenge as you'd need to build or hack a firmware image for a particular type of hardware (LTO drives are/were built by multiple vendors), and then install it in the drive. In theory it can be done but as an attack vector it is not very realistic.

If this is your biggest worry, as @Dima P. already suggested, just take out the tape and put it in a vault.

[slackuser]

I for sure understand that removing the tape from the library makes it completely immutable, what I was getting at was that it appears that the immutability of the tape while still in the robot really boils down to the drive firmware. Rob points out that its a "challenge as you'd need to build or hack a firmware image for a particular type of hardware" which is exactly my point.

So, focusing on the drive firmware, there isn't that many tape drives out there as I believe the IBM and HP ones are made by Quantum, the firmware doesn't appear to be encrypted, and if you gained admin access to the veeam server, you could for sure update the firmware.

I agree that this attack is difficult and unlikely, but so was whacking DUO when the Solarwinds server was attacked. I guess I'm just considering the difference between a very unlikely/difficult software attack vs immutability that is physically impossible due to the nature of the media or the tape physically missing. What would really make this that much more secure is a physical switch on the until that needed to be flipped to update the firmware.

As for firmware hacking, this reminds me of the old days when you can download DVD-ROM firmware to get around the region coding built in, or people changing the ROM in the game genie days. It's for sure possible.

[soncscy]

Matthew,

I am more suggesting the attack is not reasonable when you're an attacker, not that it's not possible. If you have the access to perform this attack, you have the access to perform dozens (if not more) of other attacks which are equally (if not more) vulnerable and much more readily accessible.

Think like an attacker; once you're settled in on an environment with the credentials required to perform the attack you're talking about above, what approach are you going to take?

1. Take the time to try to exploit a narrow attack vector which has very specific requirements to successfully deny your target?
2. Use the same credentials to just screw everything quickly and rapidly using normal deletes/erasing partition headers?

Both end up with the same results, but 1 requires a lot more focus, specific technical knowledges, specific technical requirements, while 2 gets the job done 99.9% of the time and gets you a ransom.

This isn't to discount these vectors, but

[Relaxe]

Sorry to revive and old thread, but I was researching exactly this and Google brang it up.

The concern is not about the improbability, but the possibility.
"LTO Drive firmware will not be hacked" is true until it's too late.
This has very real value for those criminals: erasing the backups is key for their business.

I have no tape library, and no experience, so the quick question:
For SAS connected ones I know of, there are two connections: SAS (or FC) and Ethernet.
Is the Ethernet/IP connectivity required for normal operation, or only occasional administration?
Can you operate the day to day backups and library management strictly within Veeam over the SAS/FC port?
Can you update the drive firmware over SAS/FC (It doubt it)?
If it could be possible to airgap the management while maintaining functionality, then unplugging the Ethernet cable would be a reasonable mitigation.

[Regnor]

For backup to tape jobs, only the SAS/FC connection to the library is necessary. The NIC can stay disconnected all the time, as you only need it to access the management site. Standalone drives can be updated via SAS/FC. For libraries I'm not sure, but you could still temporarily connect the NIC.

Just don't forget, that you can wipe any tapes if you get access to the tape server. Either from within the Veeam management console or directly accessing the tape drives.

[vmtech123]

Eject the tapes is the best call. It's a great task for Friday afternoons Especially if you get to drive to an off site location near your house

[Relaxe]

>> Eject the tapes is the best call
I was looking into this. Again, complete newbie without tape experience here.
Is there a way or specific library models where you can really eject the tapes?
What I see is the tapes going out of the drive and back into one of the library slots.
What I would like is the tape ejected of the drive, go into the front "mailbox" slot and push the tape out, fall in a slide to a bucket.
One backup a week, 16 slot-library means I need to service the thing once per 3.5 months when it's out of tape to chew and spit.
This would allow to use RW tapes instead of WORM to 100% protect from ransomware....
Tell me this exist!