Suspicious incremental backup size - how do you investigate this?

[kevdpc]

Sometimes I get the alert for "Suspicious incremental backup size" when I don't expect any significant changes like a large amount of data added to a VM in the job, adding VMs to the job, etc.

How do you investigate this if you're not sure what caused the large incremental backup?

Do others get these unexpectedly?

What are some other things that could trigger this alarm other than ransomware?

[wishr]

Hi Kevin,

You may use the VM Change Rate History and Job History reports to get more information about the VM size changes and act from there. You may also want to involve the owner of the applications running on this VM to get more understanding of the situation once it's understood when exactly changes that caused the alarm to trigger happened.

It may be natural for a VM to have a high change rate, depending on what kind of applications are running there. In this case, you may adjust alarm thresholds accordingly or exclude that VM from the original alarm and create a separate individual alarm for it with specific thresholds.

Thanks

[bkain1]

Interesting. I have the same issue but the suspicious backup size is on the SQL backend to the VeeamOne box. Ideas?

[wishr]

Hi Backi,

Nothing to add to my post above. Do you have any particular questions?

Thanks

[Vitaliy S.]

SQL Server is a highly transactional application, so looks like you had some activity on that server at the time when that alarm was triggered.

[dspringer]

I found this topic matching my concern and of course didn't want to make a new one.

I am currently testing VeeamOne in an unlicensed version to show the advantages of the software to my colleagues. I myself am already enthusiastic. Even though my SQL Express is already being admonished, where I only have the one main backup server with its various remote backup proxy connections and just 3 of our 10 vCenters connected *grin*.

My concern is this, which actually fits with the first post. Full and incremental backups are running. For servers where little changes, the amount of incremental file can of course change every now and then. But why is this interpreted as an error? Because in the case of ransomware, files are encrypted, so they may be recognised as 0kb and thus the total backup amount fluctuates? But what is the best way to set the warning threshold? I still have the standard with "Below" warning at 80% and error at 70%.

For example my Error was "Size of incremental backup created by "Jobname" job (31.1%) is below the configured threshold (70.0%)
Incremental backup creation time 2022-06-15 23:00:47 (UTC+2:00)"

[RomanK]

Hello David,

"Possible ransomware activity" alarm detected suspicious activity on the virtual machine by evaluating CPU usage, datastore write rate and network transmit rate. While the "Suspicious incremental backup size" alarm evaluates the size of increments. That is why I moved your post to this thread.

Over time, incremental backup points are fairly consistent in terms of storage consumption. However, ransomware could affect the size. Unexpected file system encryption or mass deletion would be easily recognized. The best way to start an investigation would be the second post, please take a look.

However, there is no best way to set up this particular alarm for everyone. You should adjust the rules of this alarm to fit your environment.

Thanks

[ValiantMartian]

Just wanted to point out this blog post and tool to identify WHAT is causing unusual incremental backup sizes.
https://www.veeam.com/blog/big-incremental-backup.html