Exclusions Don't Work When Drive is File Level Backup Target

[Carloton]

Hello, this post is intended for the Veeam Agent for Windows development team.

To begin, let me state that I opened 2 cases at the same time that both related to this issue. I am not sure which case # Veeam support specifically documented this problem in, so both case numbers are notated below:
Case ID: 02617643
Case ID: 02617631

Per Veeam support's request, I am posting this message in order to bring this problem to the attention of the development team.

Veeam Environment: Veeam Backup and Replication 9.5.0.1536 (using paid Veeam Agent for Windows licenses)
Veeam Guests: Server 2012R2

The issue (reproducible) is as follows. If you setup a 'file level backup' and then within the 'items' selection, target a drive letter (example D:\) as your backup target all 'Exclude masks' (example *.mdf) are ignored. Even though you did NOT select the backup to operate as a volume level backup, it will become one and ignore exclusions if you target a drive letter as a 'file system item' as the data to backup.

This, I sincerely believe, is completely incorrect behavior and has created extra risk for our company as well as additional admin work. We backup drives/servers for other departments and thus do not have specific knowledge and/or control over another admin creating a new folder at the root level of a drive. Thus, we always target the root of a drive for backup so that any NEW folders added are backed up BY DEFAULT rather than left out.

However, at the same time, there are certain types of files or specific directories that we do not need to backup (in the case of *.mdf files they are backed up by other means). Backing these files increases the backup time unnecessarily (by many multiples) and increases backup storage requirements, so it is unreasonable to just backup the full volume.

With the way the product currently works, we are forced to make a choice. Either backup the full volume and waste a ton of time and backup resources unnecessarily OR tediously list out ALL folders on the root of each drive so that exclusions work and run the risk that newly added files are not backed up until someone happens to notify us (never will happen) or we notice.

Being able to target a drive letter and also have exclusions work properly is a standard backup product feature that was available in our previous backup solution. It is extremely disappointing that it is missing here and was not documented in the PDF manual I followed for deployment of Veeam Agent for Windows.

Ultimately, we were forced to work around the issue by targeting all folders on the root of each drive so that exclusions will work and then setup a PowerShell script that watches all the drives on our various servers and sends an email notification when a new file/folder is added to the root of a drive so that the backup can be adjusted. We have already spent/wasted a number of hours setting this up/testing/deploying this folder monitoring 'work around' that would be totally unnecessary if targeting drive letters and using exclusions was possible.

[Dima P.]

Hello Carloton.

Thank you for sharing this issue. I'll review your case and get back to you.

[Carloton]

@Dima P.

Any updates on this issue?

[Dima P.]

Hello Carloton, sorry for the delay.

We reviewed the case and the given information looks correct: whenever you include the entire volume to file level backup scope file masks are going to be ignored. However, I can confirm that path exclusion should work if entire volume selected as a source i.e. you can add volume C to the source items and then add С:\Users\ to exclusion list.

[Carloton]

Hello Carloton, sorry for the delay.

We reviewed the case and the given information looks correct: whenever you include the entire volume to file level backup scope file masks are going to be ignored. However, I can confirm that path exclusion should work if entire volume selected as a source i.e. you can add volume C to the source items and then add D:\Users\ to exclusion list.

Hi Dima, thank you for the reply and for confirming the issue that we discovered. However, to clarify, I assume that you meant Volume C and C:\Users? In any case, I am not entirely convinced that path exclusion works even if you select an entire volume as a source. Have you personally tested this and confirmed this fact using the currently available version of Veeam Backup and Replication with Veeam Agent for Windows? I am not 100% sure but I am somewhat certain that we tested path exclusions with the entire volume selected and the exclusions still failed to work (just like file masks).

In any case, that is somewhat beside the point. Now that you have confirmed that including the entire drive letter in a file level backup makes it so that 'Exclude Masks' are ignored, what is the plan to resolve this issue? Is this going to be fixed in a future release of the product? If so, what sort of timeframe are we looking at for that fix?

[Dima P.]

However, to clarify, I assume that you meant Volume C and C:\Users?

Thank you, it was a typo. Edited my previous ports.

I am not entirely convinced that path exclusion works even if you select an entire volume as a source. Have you personally tested this and confirmed this fact using the currently available version of Veeam Backup and Replication with Veeam Agent for Windows? I am not 100% sure but I am somewhat certain that we tested path exclusions with the entire volume selected and the exclusions still failed to work (just like file masks).

Yes file path exclusion was tested by our QC team. Please let me how it goes in your environment.

Now that you have confirmed that including the entire drive letter in a file level backup makes it so that 'Exclude Masks' are ignored, what is the plan to resolve this issue? Is this going to be fixed in a future release of the product? If so, what sort of timeframe are we looking at for that fix?

While I understand the use case and it makes total sense, unfortunately, I can give you the ETA at the moment.

[Carloton]

Yes file path exclusion was tested by our QC team. Please let me how it goes in your environment.

I can confirm the opposite. I just setup a test file level backup, targeting the E:\ drive of a system. Then, under exclude masks I stipulated E:\WindowsImageBackup. After taking the backup and browsing through the backed up files, the directory E:\WindowsImageBackup was still included. How is your QC team testing this that is different from what I am doing?

[jonom]

This sounds almost like the problem I ran into a year or so ago except I was trying to do Volume level backups with exclusions.

I was told the following by support on how to get it working correctly (it is rather counter-intuitive):

To create Volume level backups with directory exclusions and/or file masks select File level backup.

In the Files section select the volumes you want to backup. When you select a volume here the description to the right should change to > volume level backup instead of > file level backup.

Now deselect the folders you want excluded and/or apply exclusions via the Advanced button. The description to the right of the volumes should stay as > volume level backup.

[Carloton]

This sounds almost like the problem I ran into a year or so ago except I was trying to do Volume level backups with exclusions.

I was told the following by support on how to get it working correctly (it is rather counter-intuitive):

To create Volume level backups with directory exclusions and/or file masks select File level backup.

In the Files section select the volumes you want to backup. When you select a volume here the description to the right should change to > volume level backup instead of > file level backup.

Now deselect the folders you want excluded and/or apply exclusions via the Advanced button. The description to the right of the volumes should stay as > volume level backup.

Hi Jonom, I appreciate the feedback. However, I am wondering if you are using a different version of Veeam Agent for Windows? Are you using the standalone version, not the version that is deployed/configured by the Veeam Backup and Replication console?

I don't have any 'description' field after I add a drive letter that shows if it is a volume or file level backup. I also don't have anywhere to add/deselect folders. All I can do in the B&R console is add folder paths manually via typing in the path. Via the B&R console, applying exclusions via the 'advanced' button is the only way to do it and it does not work properly (for folder exclusions or file type exclusions).

[jonom]

Ah, yes, I'm using the standalone VAW.

[Carloton]

I can confirm the opposite. I just setup a test file level backup, targeting the E:\ drive of a system. Then, under exclude masks I stipulated E:\WindowsImageBackup. After taking the backup and browsing through the backed up files, the directory E:\WindowsImageBackup was still included. How is your QC team testing this that is different from what I am doing?

@Dima P.

Any updates on this portion of the issue? What your QC department confirmed also does NOT appear to work in our environment. Essentially exclusions appear to not work properly in a file level backup if you target a drive letter ex. D:\ E:\ etc.

In any case, hopefully additional customers start to submit tickets relating to this issue and it moves up on the priority list for your development team, this was an incredibly frustrating issue for us as part of our initial Veeam Agent for Windows rollout using Veeam Backup and Replication.

[Dima P.]

Hello Carloton.

Thanks for sharing the update. Indeed, the described behaviout does not look like expected. I'll check with QA team and update this thread with the results of my findings.

[Dima P.]

Hello Carloton.

We can't confirm described behavior with QA team and it looks like both cases were closed. Can you please submit another case related to the path exclusion and share the case ID in this thread (we need the specific debug logs to continue the investigation). Thank you in advanced.

[yowmemperor]

I want throw my hat in here as well. I have the same issue. Veeam B&R 9.5 U4, latest agent. Any info about when this feature will be added?

[Dima P.]

Hello yowmemperor,

Sorry for the late response. If you see the mentioned behavior is you environment please raise a support case, as we need to investigate the root cause of this problem. Thank you in advance!

[AlessioAP]

Case 03549758

Hello guys,

after some investigations from Veeam Windows agent team, It was understood that the issue is is most likely a VSS limitation (related to shadow copy optimization writer not being able to remove those files from snapshot in time).

Hope it helps you.

[Dima P.]

Hello and welcome to the community forums Alessio.

Indeed in most of the cases this problem might be related to the VSS writes: if the huge amount of files is being excluded VSS writer might not process all the list of exclusions and eventually time out during the exclusion step of the backup. Thank you for updating this thread. Cheers!

[LucaMCT]

Case 03712030

Same here, I use file level backup and "operating system option" as "directory to backup"; exclusion list is totally ignored.
In the specific case, the file to ignore are less than 10, all in the same folder and with a unique file extension (VHDX).

[RseJonathan]

I am also having a nearly identical problem as the OP. I've done some tests around this and they seem to verify the hypothesis that this is an issue with VSS not being able to remove the files or something similar. I set up a separate volume on the same server and put only a few files/direcories on it, then excluded one and did a file level backup of the root of the volume. The exclusions worked as expected but they fail on a much larger volume with many files/directories. Is there any workaround for this behavior? If it is a timeout issue waiting for VSS, is there a way to increase the maximum wait time?

[alielnagar]

me too
i have a problem the specify (include content) ignored totally
withe version 9.5 and 10
please help me to solve this

[1971merlin]

Same issue here for me. 25Tb drive (we are backing up in file mode, indexing the file system, no virtualization (physical box), backup is the "root" of the drive - in this case I:\

I need to exclude the bulk of data (which is raw video files) and just capture some other related data which is 5% of the total drive capacity.

Excluding the video file types seems to be totally ignored, and it backs up everything, leading to an ~25Tb backup file being created, which in turn fails due to filesystem max file size limitations. The backup progress information doesn't mention anything about filenames or folders being processed, so it's really hard to understand what is happening.