Real-time performance monitoring and troubleshooting
Post Reply
WinstonWolf
Veteran
Posts: 284
Liked: 11 times
Joined: Jan 06, 2011 8:33 am
Contact:

Dealing with the "possible Ransomware" Alarms

Post by WinstonWolf »

Hi there ,

Sometimes i become the Supicious incremental Backup Size Alarms and Network Transmit Rate Alarms . Both are back to normal in Seconds .

Alarm: Possible ransomware activity
Status: Error
Previous status: Reset/resolved
Time: 17.09.2021 04:15:55
Details: "Network Transmit Rate" (40.0 MB/s) is above a defined threshold (40.0 MB/s) for '4000';
"CPU Usage" (95.6%) is above a defined threshold (80.0%)

and at the next Minute the Alarm is resolved. :

Alarm: Possible ransomware activity
Status: Reset/resolved
Previous status: Error
Time: 17.09.2021 04:16:15
Details: All metrics are back to normal

I think 1 Minute is too fast for trigger this kind of Alarm . It produces so many Emails .
Is there an Workaround for this ?

Is there an good Workaround for the Ransomware Alarms . The fill my Email Account full in one night.

Thanks
Michael
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Dealing with the "possible Ransomware" Alarms

Post by wishr »

Hi Michael,

You should adjust the rules in both of these alarms to fit your environment. Basically, these two alarms require fine-tuning. Please, keep in mind that you may also configure different rules for different servers by creating alarm copies and modifying the assignment if a certain sever constantly causes an alarm to trigger and it's proven not to be caused by malicious activity.

These alarms are very important! Recently we had a story when a customer was hit by ransomware and he ignored these alarms thinking they were false positives. Unfortunately, they were not...

Have you any questions - let me know.

Thanks
WinstonWolf
Veteran
Posts: 284
Liked: 11 times
Joined: Jan 06, 2011 8:33 am
Contact:

[MERGED] Best Practice - Suspicious incremental backup size

Post by WinstonWolf »

Hi there ,

The Alarm "Suspicious incremental backup size" drives me crazy .
On Friday we configured our weekly synthetic Full Backup for every Job . On Weekend we do no Backup to Disk Jobs. But on Monday Veeam One told me that the incemental Backup Size is very differnet from the Friday one . And yes its correct. But Veeam one Alarms me .
A colleague creat on Sunday Evening an Quick Backup because of an Windows Update of the same server . And again Veeam One creates an Alarm with "Suspicious incremental backup size" because its again different to the Friday Synthetic Full Backup .

Alarm: Suspicious incremental backup size
Status: Error
Previous status: Error
Time: 19.09.2021 18:43:23
Details: Size of incremental backup created by "VMDK_XXXX" job (35.6%) is below the configured threshold (70.0%)
Incremental backup creation time 2021-09-19 18:34:57 (UTC+2:00)

What is the best Way to handle this Problem . Sometimes the Backup Time on an Backup Job is also longer because of weekly Backup Files health check from an Job .

Thanks
Michael
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Dealing with the "possible Ransomware" Alarms

Post by wishr »

Hi WinstonWolf,

I've merged your new post with the existing topic you created to keep the context for future readers.

As mentioned above, you should tune the alarm based on the specifics of your environment. This includes modifying various settings in the alarm rules including analysis depth and alarm thresholds. Could you please share a few screenshots of your current alarm rules config?

Thanks
WinstonWolf
Veteran
Posts: 284
Liked: 11 times
Joined: Jan 06, 2011 8:33 am
Contact:

Re: Dealing with the "possible Ransomware" Alarms

Post by WinstonWolf »

Hi ,
Here is the Screenshot from this Alarm . At the Moment with the default Settings .

Image


What means the Percent values ?
What means "Suppress for the restore point following a job scope change ?


Michael
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Dealing with the "possible Ransomware" Alarms

Post by wishr »

The percent values represent alarm thresholds, i.e. define in which conditions the alarm should trigger.

The "Suppress for the restore point following a job scope change" checkmark does pretty much what it says - defines how the alarm behaves when the size of the restore point changes due to a change to the list of objects added to the job. This setting is only applicable for jobs using per-job backup files. If you are using per-job backup files (i.e. not using per-VM backup files) this is a go-to option.

Another important setting is "Analysis depth" which controls how many previously created incremental restore points must be analyzed using the "Relative" detection approach. I suggest playing with this setting to see what value works best for you.

In the cases similar to what you described above when your colleague created a quick backup due to a Windows update that caused the incremental restore point to be of very low size compared to the previous incremental RPs, you may:
1. Just check what caused that situation and acknowledge the alarm if there is nothing suspicious going on.
2. Lower the alarm thresholds in the second rule. Though, keep in mind that this change may lower your ability to detect some real ransomware.
3. Increase the value of the "Analysis depth" setting, play with various values to find out what value works best for you.

Hope it helps! Let me know if you have any other questions.
Post Reply

Who is online

Users browsing this forum: No registered users and 9 guests