Delete files, folders from within Backups

[davecla]

Hello Team Veeam,

Is there anyway to delete files / folders from within backups - leaving the rest of the backup intact?

Have semi frequent need for remove selected information from backups (court ordered).

Any ideas?

Ideally it could be actioned programmatically and I could get a dev to wrap some code around to automate the process.

thanks
Dave

[Gostev]

Hi, Dave.

Basically, you can exclude files and folders during backup, but not after backup has been created (since it's an image-level, and not a file-level backup).

Best you can do is "ugly workarounds". One way to achieve this would be to spin up each restore point that you want to keep with Instant VM Recovery, delete the required files from the running system, and make another backup of this instantly recovered VM (for example, with VeeamZIP). All of this can be done programmatically. Once you're done creating the restore new backups, you should simply delete all the original backups.

I will take a note of this use case for the future versions (zeroing out content of selected files in all restore points). In theory, it should be doable - but it is not an easy feature to implement for sure. Also, the whole process will probably still require a very long time to complete (as we would need to mount each and every restore point) - but, at least it will be fully automated.

Thanks!

[chrisdearden]

I suppose its the opposite of "legal hold" - more of a "legal purge"

[Jekit]

Hello
Do you have news about this request ?
Our servers host data from multiple clients.
We would like to delete a specific folder (example: SERVER / DISK / Client / XXX)
Is there a Veeam tool or partner product to help us automate this?

[foggy]

Hi Jeremy, you cannot delete any data from the backup files. But could you please elaborate on your request - is the "right to be forgotten"-related by any chance?

[pesos]

We are in the same boat as the OP - facing a legal issue where documents were disclosed in error to our legal team and now need to be purged from our systems entirely. Any update on this kind of functionality?

Thank you.

[Gostev]

No updates John. There has been very few requests for this functionality in the past years, as you can see from the dates of posts this thread. Thanks!

[pesos]

Good to know. Definitely something to keep in mind when deciding whether or not to use per-vm backup files!

Thanks!

[it2018]

Is there any feature, or planned feature to be able to selectively delete files/folders from a backup or backup chain.
Occasionally the need arises due to client contract requirements that we remove all data after the project completion.

We're currently moving to using Veeam, and with our current software we would have to exclude the folder at the start then create a new job specifically for that. Does Veeam have any way of deleting from the backups rather than having to do this exclude/separate job method?

I appreciate it's a slightly odd requirement to want to delete from backups though I would think possibly more common with GDPR rules these days.

[wishr]

Hi Chris,

You can modify your job and exclude certain folders/files/file types from a job at any point, but not from the existing backups.

I've moved your post to an existing thread covering the same topic - please take a look.

Regards,
Fedor

[cookiejc]

HI, I Just wanted to ask the same question really and found this thread. We work in government and use Veeam, with various government rules around data handling and the newish GDPR regulations, being able to remove individual files from existing backups would be a key requirement when we re assess our backup solution. I imagine every organization will face this issue. What do you do if a right to be forgotten request requires you to delete individual files from backup relating to an individual?

[Gostev]

Hello, backups themselves cannot be modified due to being image-level. However, even with file-level backups, attempting to remove individual records from within application database files on the binary level will corrupt the database - while removing the entire database file is rarely an option.

However, GDPR does not require removing data from backups either. And in fact, there are other existing regulations which require backups to be immutable. Otherwise, for example, criminals would be able to just erase the history of their crimes by requesting to remove their name from all records including ones associated with money laundering, making further investigations impossible.

What a GDPR's right to be forgotten requires is that personal data is not restored from backups back to the production environment, where it will become the part of production workflows like marketing emails. For this, we provide Staged Restore functionality to make required modifications to the backups, removing sensitive data from files and application databases before restoring the modified state to the production environment.

Here's a good video about GDPR > https://www.veeam.com/videos/general-da ... 11236.html

[cookiejc]

That's useful information thanks for responding

[Steven Bricklayer]

Any update about that request?

We have needs from clients to be able to delete all the database and data we have about them.

The staged restore solution is very complicated for such a basic need.

It means we will need to create one machine per client?

Thanks

[foggy]

Hi Steven, could you please elaborate on the actual requirement? As stated above, there's no requirement to remove data from all backups. If we're talking about Staged Restore, then you are able to perform it for several VMs within a single restore session. Thanks!

[Steven Bricklayer]

Hello,

Exemple :

I've a MS SQL server, hosting 5 databases.

I've a retention policy of 10 years.

3 years after I need to delete ONE of the database, from all our backup.

How am I supposed to do it?

Thanks

[soncscy]

Hi Steven,

I think you just need to rethink your backup strategy as it's inconsistent with the obligations you've opened yourself up to with clients.

Image level backups are all or nothing, as you can see, and for sure I hope that no such feature to delete data from an Image level backup at a whim is ever implemented. It's a nightmare for the regulations I have that require that the image level backups be unalterable in such a matter. It's why we chose image level backups instead of traditional Guest-Level Backups.

You want DB level backups here, not Image level backups, and I don't see a lot value trying to get Image level backups to do that sort of thing as the focus is completely different. It's like trying to pound in a screw with a hammer -- surely you can do it if you hit the thing hard enough, but you're going to end up causing a lot of needless destruction, when you could just go and buy a screwdriver.

If I were you, I'd simply do the following:

1. Script periodic native SQL backups of your client databases to dedicated directories on the SQL server
2. Set up some Veeam job to backup, and only include those directories in the backup (lots of options for this)

I'm not sure you get the Veeam application restore options this way, but you get your granular deletes.

[gairys]

I actually have a specific need for this, with a very specific example. We run an on-premise Exchange server for a few researchers who work with government contracts. Sometimes these researchers "forget" to use the approved methods of file sharing and send an email with a sensitive document to a colleague at another organization. In this case, we are required to go into the Exchange server and remove all traces of the sensitive document, work with the receiving organization to purge their copy as well, then locate and purge all traces of the document from our backups.

For years we have been using physical servers for our Exchange environment and DPM for backups. Our DPM admin has a way to mount the archives and remove the data and provide evidence that the data has been removed. Recently we have been discussing moving our Exchange servers to Hyper-V, and we use Veeam BnR for all Hyper-V and VMware backups. Not having this feature throws a wrench in our process if we can't purge specific data because we would have to purge the entire backup.

We have worked around this in the past by using multiple databases and breaking out each into it's own backup job to minimize the amount we need to purge. Being able to mount and purge would help tremendously in our environment.

[soncscy]

Gairy,

Can you share the regulation that governs this and its actual text? This is mostly for my own curiosity since every once in a blue moon I get a requirement from clients we're on-boarding with similar conditions, and every time we've asked for the actual regulatory text, either silence or it turned out to be a misunderstanding.

For the regulatory items my team deals with, backups are usually handled differently with the requirement for purging such breaches/sensitive items, as for example, if you have a regulatory requirement to purge data but also to have immutable data, these requirements are naturally at odds. Or suppose an item is declared "must be purged" but the server is on 200 different tapes somewhere in a vault in a mountain? Is it really required to retrieve and delete all of these tapes as they have "must be purged" data? I don't have a ton of faith in tech regulatory bodies, but surely they at least considered how messy and expensive this would be.

The Staged Restore feature of Veeam is something we've successfully used; basically, we maintain a scripting tool that on boot, checks a database we keep for "must never be recovered data", and runs a bunch of checks for such data on servers considered "sensitive", i.e., they might contain the data.

The script + the db to maintain are trivial to write up, and by restricting who has restore access and monitoring all restore activity, we satisfy the requirements readily.

This is why I'm curious on the specific regulation as almost always I see claims like this, but rarely does the actual regulation require such behavior. I'm not saying you aren't being accurate, I more just want to know what org actually requires this. If it's truly something you need, then from my point of view it's the same as I responded above: image level backups are not suitable for this requirement, you need to dump the EDBs periodically. A long as you have the logs as well, you should still be able to do the required manual deletion and still utilize Veeam to do the restores. But it just means you need to not use image level backups and instead do database dumps.

[hke]

Another use case I didn't see mentioned - removing malicious files (or registry keys or other objects).

It would be very nice to be able to surgically remove artifacts from a restored VM before it is booted.

[Gostev]

Good news, our Staged Restore functionality does exactly that see my previous post.

[BackupBytesTim]

Note that I don't actually have a particular need right now to do this, but I am curious about it as this is another feature that is commonly available in other backup software, such as Acronis, so I'm curious about the explanation:

backups themselves cannot be modified due to being image-level

As it was explained to me, Veeam does utilize the system's snapshot capability to create a point-in-time state of the contents of a volume, however it does not actually back up the snapshot. It creates the backup file using the contents of the snapshot, so everything is still stored in the backup as individual file contents that are distinguishable, which is also how the Staged Restore feature is able to exclude some items from a restore, or how individual files can selectively be restored from a whole volume backup.

So, again while I don't actually need to do this right now, it has come up from time to time that a particular restore point has some significant increase in data due to some extra files that were temporarily present on a system being backed up, but are not desired to be retained and make the entire backup chain unnecessarily large. Our normal resolution to this is to do a new Active Full backup and delete the entire previous chain before its normal expiration date, however obviously that's less than optimal since it results in a reduced number of restore points until the new chain becomes long enough to meet the desired retention policy.

So, I'm not understanding why individual files couldn't be deleted from a backup chain, whether across all restore points or even just a particular restore point (which is what I'd like the ability to do).

If it's just a matter of the feature isn't there because no one developed it, that's one thing, but otherwise I don't see how the "image-level" explanation is actually an explanation.

[Gostev]

As it was explained to me, Veeam does utilize the system's snapshot capability to create a point-in-time state of the contents of a volume, however it does not actually back up the snapshot.

Actually, either there was some misunderstanding, or it was explained to you incorrectly: Veeam does in fact backup the actual snapshot. Which is that very "image" from when we're talking about doing "image-level" backup. A static, point-in-time state of each machine's disk in a form of a raw disk file.

Fun fact: regardless of protected workload and the type of source disk (physical disk, VMDK, VHD, VHDX, QCOW, QCOW2, etc.) it is always a RAW disk file inside of the backup file. Never a platform-specific format. Which, in particular, helps with granular and cross-platform restores.

[BackupBytesTim]

Dima P. said here: post498067.html#p498067

Snapshot is created on the machine and is used for backup however we are not transferring snapshots, just a note to be on the same page.

Could you two discuss that and clarify who is accurate? Or have someone else share their input on the matter, when two Veeam staff seem to be giving contradictory information I like to get a third opinion, or at least have it be clarified.

Fun fact: regardless of protected workload and the type of source disk (physical disk, VMDK, VHD, VHDX, QCOW, QCOW2, etc.) it is always a RAW disk file inside of the backup file. Never a platform-specific format. Which, in particular, helps with granular and cross-platform restores.

I'll admit that I haven't actually tested it lately, but my understanding was I can't restore data from a backup created by one operating system to another operating system, is that not accurate? Similarly my understanding was also that I can't restore a VM backup from a Hyper-V server to a physical computer, or vice-versa, is that also inaccurate?

[Gostev]

Both of us are correct, just talking about different products and different backup modes even:
1. I was answering the thread about Veeam Backup & Replication and its image-level backup capabilities.
2. Dima P. was responding to the thread about Veeam Agent for Windows and more specifically, about its less popular and non-default "file-level" backup mode (image-level backup is still the default).

Restoring data from a backup created by one operating system to another operating system is certainly not a problem. Ultimately it's like copying a file from Windows to Linux, so why would there be any problems?

As for your question about entire machine restores, the answer depends on a backup product. For example, Veeam supports "ANY to ANY" cross platform restores, with just a few exceptions. In other words, you can restore an image-level backup of any physical, virtual or cloud machine to a VMware, Hyper-V, AHV, AWS, Azure or GCP VM. Check out this slide from some internal presentation.

[BackupBytesTim]

Ah, I guess I took that explanation to apply to everything across all Veeam products. I didn't think it was specific to the Windows Agent. Not really anyone's fault, but I'm still not fully adjusted to Veeam's different components, my previous long-term experience was with Acronis where there's literally one application that you use to back up everything in any scenario, so I still sometimes forget that Veeam has multiple components and they don't necessarily function the same way.

Regarding cross-platform restores, that's good to know. I'll admit I can't recall where I saw something about Veeam not supporting that, but it's good to know it can in-fact do so.

Thanks for clarifying.

[Gostev]

For the avoidance of doubt, Veeam Backup & Replication is that "literally one application that you use to back up everything in any scenario" you're looking for. This is your one-stop shop for protecting any single enterprise environment.

Some of it's components CAN be used standalone as well, but you don't have to unless your use case calls for doing so. For example, it might be an overkill to install a Veeam Backup & Replication server in an environment with a single physical server to protect.

Regardless, you can expect all Veeam Backup & Replication components to provide the same base level of identical capabilities for identical workloads (e.g. for machines). An example of such capability is image-level backup, which is available for all machines: physical, virtual, cloud.

Some components may also provide additional unique features thanks to their design, for example having a backup agent installed on a protected machine enables us to take agent-based file-level backups, which are generally not possible with host-based image-level backups (where a backup application accesses a disk image from the outside, does not typically have a knowledge of what is stored inside of the image, and thus only able to backup it as a whole).

Hope this clarifies it a bit.

[BackupBytesTim]

I'm aware this is getting a little off-topic at this point, but to clarify the "back up everything in any scenario" comment, does that mean I can use VBR to back up the server it's running on? My understanding was that VBR could only back up VMs via their hypervisor, so it couldn't back up it's own server unless it happened to have outside access to said server on account of that server being a virtual machine where the hypervisor was accessible from the VBR server.

If so, that's great, I could be using that in place of the agent software (which is 90% of my customer-base) so that I can have better remote access to said computers without needing third-party software as well, but it was my impression I couldn't do that.

[Gostev]

Agree, let's not derail this particular conversation any further. Feel free to create a new dedicated topic, describe in more details your Veeam deployment scenario, what are you trying to achieve, and we can discuss the best way to achieve this there with the right PMs. Thanks

[RubinCompServ]

To return back to the original question:

When backups are mounted for us with the various Explorers, that data gets presented in a user-readable format (obviously) and, presumably, as read-only. Would there be any way to configure some type of "Advanced Mode" on the Explorers that would allow the presented data to become read-write? Perhaps even, instead of actually removing data from the backup, "simply" writing a tag of some sort that says, "this data should not be restored"? I know that that can be manually handled via Staged Restore (as mentioned above), but that becomes cumbersome to keep those lists and have them accessible when performing a restore, whereas having that tag set at the time of customer submission means it will always be there. Even if all it accomplishes is to build a script for Staged Restore to automatically remove that data (akin to an Answer File).