Malware Detection - too many files have had their names changed

[tomtom94]

Hello!

The last few days we got warnings from Malware Detection:
Detection source: Bulk file renaming
Status: Suspicious
Details: Potential malware activity detected: too many files have had their names changed since last backup, ensure they were not encrypted by ransomware

The server runs remote desktop services and therefore a lot of users.
Does Veeam Backup log which files have been renamed so we can check who/what causes this behaviour?

I tried the Support Collection Wizard but in the exported log files i cant find any information regarding malware scans.

with kind regards
Tom

[Dima P.]

Hello Tom,

Starting from version 12.1.1 we do. We've added a dedicated log which is created whenever such event is raised. Please update at your own convenience and let us know if it helps you with the investigation. Thank you!

[tomtom94]

Hello Dima!

Thanks for your fast response but im still struggling to find the logs.
We are allready running version 12.1.1.56. (updated last week)
The release information states that: "The location of the corresponding log file is now displayed directly in the session logs and the event details."
Well i must be blind ... i cant find it.
The folder "C:\ProgramData\Veeam\Backup\Malware_Detection_Logs" is empty and in the backup logs i cant find any information regarding malware scans.
Any further tips ?

best regards
Tom

[Gostev]

Could you perhaps share a screenshot of the event for Dima? So he could see what you're seeing.

[tomtom94]

Hello!

Sure here is a screenshot of the potential malware activities.
As mentioned before there are tons of logs in "C:\ProgramData\Veeam\Backup" but "C:\ProgramData\Veeam\Backup\Malware_Detection_Logs" is empty.
And sadly there is no information displayed if or where the logs are located.



Best regards
Tom

[Dima P.]

Hello tomtom94,

Thank you for the screenshot! Looks like for 'Bulk file rename' event we do not have a dedicated log, will discuss with the team. As a workaround can I propose to compare files in the restore point against your production machine via compare functionality?

[tomtom94]

Hallo Dima!

Thanks for your suggestion but without knowing which files have been renamed i cant compare them.
In this example there are 1.900.000 files in 320.000 (sub-)folders ...
Well im pretty sure its just a false positiv and therefore i added this server to the global malware exclusion list. (And yes i did some random sample surveys, compared files, checked some folders but havent found anything suspicious)

Please submit some suggestions to the team:
1. Log at least the folder so we know where to start.
2. Top notch would be some kind of file browser where you see the differences between two backups or production/backup, with markers like renamed/deleted/moved/how many files have changed/etc. (like the actual backup browser -> compare with production but a lot improved)
3. Add some options where we can fine tune thresholds. (how many are to many, maybe a percentage slider, profiles for type of server (RDS/fileserver/...) etc.)
4. Add option to exclude folders in vm´s. (not just extentions)

Best regards
Tom

[mriesenbeck]

I have several patched instances where the logs are ending up in the malware folder, but one where it doesn't. I thought that the issue was that we changed the logfile location in the registry, but changing back to default does not fix it. So I'm going to create a case.

[Dima P.]

Hello,

1. Log at least the folder so we know where to start.
2. Top notch would be some kind of file browser where you see the differences between two backups or production/backup, with markers like renamed/deleted/moved/how many files have changed/etc. (like the actual backup browser -> compare with production but a lot improved)
3. Add some options where we can fine tune thresholds. (how many are to many, maybe a percentage slider, profiles for type of server (RDS/fileserver/...) etc.)
4. Add option to exclude folders in vm´s. (not just extentions)

Noted, thank you for the detailed feedback!

I have several patched instances where the logs are ending up in the malware folder, but one where it doesn't. I thought that the issue was that we changed the logfile location in the registry, but changing back to default does not fix it. So I'm going to create a case.

If you are referring to the log dedicated for 'Bulk file rename' event there is no need to create a case as such log currently does not exist. We've noted a feature request for this functionality. Thank you!

[cgsm]

I also have this issue with a Remote Desktop Server. I think the issue, at least to me, stems from all the user-session related files being changed. I.e., each time a user logs in, Windows changes a bunch of files and VBR catches this. I have simply turned of Malware detection for this VM. A bit annoying since this VM is where my users will most likely download malicious files.

[Dima P.]

cgsm,

Thank you for the feedback!

all the user-session related files being changed

Is there a location on this Remote Desktop Server's file system which you suspect to cause all the trouble or these are random changes?

[zhallman]

Hello,

Similar issue here. Case #07107057.

I keep getting detections that I believe are flase positives, but I can't seem to find for information about the alerts to verify.

I get the 2 following messages:
Bulk file renaming Potential malware activity detected: too many files have had their names changed since last backup, ensure they were not encrypted by ransomware

Ransomware note Potential malware activity detected

I can't seem to find anything that gives me any files names or any more information. I checked the log files and that did not seem to help at all.

I did check the server in question manually and did not see any unsual behavior or files.

Veeam engineer reply:
So unfortunately, this specific event, bulk item renaming, currently does not generate a log. This seems to be an oversight on our part as we expected it to based on latest release notes, but we're currently working to resolve this.

Looks like it is not avaiable in the current version best options are to exclude or manually compare

Thanks

[Dima P.]

Hello zhallman,

Correct unfortunately there is no dedicated log for Bulk file rename event, I've added your vote to the improvement request. Thank you!

[snorma01]

I would like to chime in support for a solution for Remote Desktop Session Host (RDSH) servers specifically, which doesn't create daily false positives for bulk file renaming. As was previously mentioned, this is the highest risk server in our entire environment, so other than perhaps file servers this is the among the most important type of server to keep malware scanning enabled on.

[Dima P.]

Hello Stephen,

Just to make sure I got this right: you vote goes to excluding the specific file locations for such servers, right? Thank you!

[damiengm]

Hi,
I'd like to add my vote to have a log for this "too many files had their names changed" on my Windows server using the Windows Agent. Oddly enough this Windows server is the Veeam service server (has not too much else on it).. spot checking there doesn't seem to be a problem, but without knowing where to start is an issue.

[Dima P.]

Hello damiengm,

We plan to add such troubleshooting information as soon as we can. Thank you for the feedback!

[pmichelli]

+1 for a log for this. One of my servers started doing this last Sunday (coincidentally the day the clocks changed) and it's really annoying to not know what files Veeam thinks were changed

[Dima P.]

Thank you for the feedback pmichelli! Do you suspect any particular activity / event to cause this?

[pmichelli]

Hi Dima,

We cannot figure this out. The team that owns the VM went over it twice, I logged in and poked around and could not see anything out of the ordinary.

My security team looked at the SIEM and could not find any suspicious activity on that day.

This is a test ADFS instance that is not used often. If I had to guess (since no other VM is doing this)

1: Changing of the clocks to DST changed timestamps on some (many files)?
2: Some Windows update that ran (You never know what MS does with these things)

It is odd that it only flagged this VM. I marked that backup as clean and it has not come back (did not exclude the VM)

[Dima P.]

Understood, thank you!

1: Changing of the clocks to DST changed timestamps on some (many files)?

Doubt that this is the case buecause index analysis does not track the attributes, only actual file paths.

2: Some Windows update that ran (You never know what MS does with these things)

Yup, could be the reason.

Hopefully once we release the detailed logs we can sort it together. The functionality is on it's way, so stay tuned for the next minor update!

[pmichelli]

Hi Dima

I have more info. It triggered another alert except this time it was for too many .txt files deleted, which had a log. It seems our patch manager Ninja is creating a ton of .txt files and then purging them a few days later. I opened a SR with the vendor.

[Dima P.]

Hi pmichelli,

You can change the value for txt files in the trackedfiles.xml, please take a look at this post. If you do not want those to be tracked for mass deletion event you can go ahead and remove the entire node from the xml file. Thank you!

[Joris360]

We are getting the same 2 events (ransomnote and deleted files), but don't know where to start looking. This is mostly on RDS environments as well:

so +1 for following:

Log of of the folder (or files)
fine tuneable thresholds
Exclude specific folders

[cme_b2b]

I'd like to add to this discussion by asking if there is an option to disable the file renaming/deletion check on a server/vm instead of disabling the whole scan itself. If not, please add this in the future.