Delete files, folders from within Backups

[Gostev]

"Not restoring" something is not really an option when talking about image-level backup and restore. Which is why Staged Restore is the only technologically viable approach. Only the specific application itself knows how to correctly handle the content of its database, which is why the application needs to be made running and then instructed to erase sensitive data, during which process you can indeed think of your backup being "read-write". After which, the actual (post-cleanup) machine state can be placed back into production environment.

[BackupBytesTim]

Not to argue the methodology of how Veeam is designed, but the explanation just doesn't make sense to me. It seems like it's just not a feature, not that there's a technical problem with making it a feature.

If we assume a simple disk image, well if I make a disk image with other software from a disk of my computer, I can then edit that disk image, removing or adding things to it, then I can write the disk image back to a physical disk. So the explanation that a disk image can not be modified is not accurate, maybe Veeam can't do it, but that's just then "not a feature", not "technically impossible".

If we assume Veeam's data is stored in Veeam's format (which, it is, that format may involve one or more "disk images", but it is a Veeam proprietary format) (unless that's totally wrong, in which case, do correct me on that), but then since Veeam created the format and Veeam's software added data to the file, why can Veeam's software not also modify the file? Seems again to just be a lack of a feature, not something technically impossible to do. My understanding is Veeam doesn't even have a feature to do actual raw disk images, and even if it did it should still be able to read the contents afterwards provided it was formatted in a way Veeam understood (such as NTFS). But even with a not-raw disk image, ultimately the disk has partitions, formats, volumes, files, metadata, all of it things that Veeam can read and write, so it should be fully capable of modifying the contents of the file as much as it can selectively restore only parts of the backup file to the disk.

I do understand the explanation Gostev gave regarding "specific application(s)" however, outside of specific database contents or Active Directory elements, other app specific things of that sort, the disk ultimately holds files, and if a human wants to remove files from the backup that should totally be doable, now if a human deletes some file that renders an app-specific database restore impossible, well that's the user's fault, but it shouldn't mean Veeam doesn't allow modification of the backup file entirely. Other software can modify backup files just fine, it would be nice if Veeam could do the same.

[Gostev]

Let's approach this differently then. How about you share an example of other image-level backup software that is able to modify guest files directly within an image-level backup then, and we go from there? Because there are a number of pragmatic issues such software would need to solve. For example, I'm naturally curious how it is possible for such functionality to play nicely with incremental block-level backup. When you would be effectively modifying "the past" at the point when "the future" has already happened and is stored as a delta from the unmodified past.

[BackupBytesTim]

My specific regular comparison for Veeam functionality is with Acronis, which does offer the ability to remove past versions from the backup chain, it removes entire versions (incremental, full, or differential backups along a chain, everything captured at the time a backup was performed), not just specific individual files from that backup "version", but functionally I imagine that part doesn't make a difference to how it works with block-level backups. I assume it keeps track of what blocks are associated with what files in each backup "version". So I can remove an entire version and it just deletes the data used exclusively in that version, if certain blocks are needed for recovery of other versions in the chain then those blocks are not deleted. I assume. Honestly I never questioned too technically how it works because it always did what I wanted it to do.

So if I were to explain how I would make that work with modifying the "past" when the "future" relies on data from the past, it'd be just to not delete blocks that are needed for recovery of a future point in time. I'm assuming Veeam already tracks what blocks from what VIB files are needed for recovery, to my knowledge it doesn't restore the VBK, then restore the entire contents of each VIB file up to the desired point in time, so to some extent it must know what data belongs to what files at each point a backup "version" was created.

Similarly though Apple's Time Machine (which is in my opinion the only good backup software for macOS, I wouldn't highly recommend anything else, not Veeam, not Acronis, nothing) can actually delete specific files from specific "versions" from a backup file. And of course if I just clone a disk to a VHDX file or something I can mount it and edit the contents same as I would an actual physical disk.

Side note, the Time Machine recommendation is not a fault of Veeam or any other backup software, it's really just due to quirks about Apple's security that make any other software much less useful in that regard (without significant changes to security settings and very specific procedures and limitations for performing the recovery), I don't expect Veeam to make itself better there, I do understand entirely the issues are with Apple's security preventing other applications from just having full disk access on their own or being able to boot up from recovery media without disabling things, among other things.

[Gostev]

I think I now understand the key piece you're missing: the process of deleting a file from disk does not actually touch disk blocks backing said file. They remain as is and still contain the file's content, which for example makes it possible to "undelete" files. This is the case with pretty much all modern file systems.

When you delete a file, the only thing that is updated is its record in the MFT (Master File Table). You can actually think of MFT content as a "specific database contents of a specific application" from your previous post. It is a complex structure that can contain thousands of file records in the single disk block backing that part of the MFT.

So even if a software does manage to orchestrate individual file deletion from an image inside backup and "save" the modified image state back to the backup, which in case of a single file deletion would mean saving just a single disk block backing the affected part of the MFT, the consequences will be... interesting.

For example, the very next (already created) incremental backup will most likely contain a different version of the same disk block again, but obtained from production storage in the world where the file in question was NOT deleted. So if you restore from this restore point, your file will magically reappear - as again, its actual content stays intact on disk when you perform a file deletion.

And that's the easiest scenario. The other are potentially much worse. Say, the disk block backing the modified MFT part is not replaced by the following incremental backups, however they will certainly replace a bunch of other MFT blocks taken from the world where the file in question was NOT deleted. This means the resulting MFT would be "glued" from disk blocks taken from two different worlds and will mostly likely be inconsistent, with totally unpredictable consequences if you were to restore such machine - for example, the whole volume might fail to mount due to the file system driver seeing an MFT checksum issue or a USN inconsistency.

Of course, all these issues are specific to image-level backup only. For file-level backup, deleting a particular file from backup should be a trivial task.

[BackupBytesTim]

I do understand how all that works, but I don't understand why any of that affects the ability to delete a file, it sounds like it would only be a problem if the software deleting the file doesn't do it properly...

I certainly understand if I were to delete a file from the backup, and it's completely gone, no record of it anywhere, and then I back up the computer again, and the file is still on the computer, and so it gets readded to the backup in the next version, then if I restore a new version of the backup (newer than where the file was removed from the backup) the file will be restored, because it's in the backup.

But your scenario where you mentioned the "block backing the modified MFT part is not replaced by the following incremental backups" sounds like that's a bug in the backup software then, whichever software is being used, Veeam or otherwise. I would think the MFT is an important thing in full disk image backups, does Veeam not back it up each time? I guess I don't see why that would even be a problem that needs to be considered. I mean, if we're going for "What happens if the backup doesn't contain every block needed for proper recovery?" then there's lots of other issues I could bring up too... but I always assumed it just backed up everything that changed, or just everything regardless of changes, depending on the settings for that particular backup job occurrence.

Following the same logic, if I delete a file on my computer, yes, typically the file's contents are left on the disk until they're overwritten for one reason or another, but Windows and the filesystem handles that properly, if I reboot the computer or something the file doesn't just "magically reappear" for any reason, so I guess I don't see why Veeam wouldn't be able to modify the backup file the same as Windows modifies the live disk.

[Gostev]

Veeam or any other backup software will not backup blocks which were not changed since the last backup run. This is obviously not a bug, contrary to that its the only right way to do incremental image-level backups

Now, if you seriously suggest backing up the entire MFT each time, then you should educate yourself about it's structure and what it contains... at which point it should become clear why it's a really bad idea