Yes, Ransomware can delete your Veeam backups.

Availability for the Always-On Enterprise

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby Mike Resseler » Wed Mar 29, 2017 5:57 am

Gerald,

The authentication details are stored in the DB and certainly not easily readable. That data in the DB is encrypted and you cannot just decrypt that. But that being said, a very specific written Trojan might succeed in getting that information out and then access the NAS and encrypt your backup files. Far fetched? Maybe, possible, unfortunately.

When you backup to a Veeam server, it is still the client who manages retention. After all, in the client, you can change the retention from 7 days to 1 (for example) and the change will be done on next backup.

But if you read through this entire thread (I know, it is a lot :-)) you will certainly find good mechanisms to defend your system. If you consider backup to a VBR repository, then you can do backup copy jobs of that data to a repository that supports rotating drives.

As you will see in this thread, you can harden your solution as much as possible, making the possibility of losing your backup files much less possible, but as long as it is not air-gapped, there is a risk. At my home (for the family) I simply use rotating USB devices (and the nice little checkbox to eject the device after backup). The only thing I had to do was teach my family (make that the kids, they understood it faster :-)) that each day they need to unplug the USB, and plug in the other one. Yes it is manual work and it is not fault-proof but it could be that simple :-)
Mike Resseler
Veeam Software
 
Posts: 2638
Liked: 315 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby infused » Thu Mar 30, 2017 1:32 am

I use NAS's to store backups for this very reason. I suspected something like this might start happening.

The admin account is disabled, with another one I use. There is a Veeam account which has RW to a folder, plus another one for retention on another NAS.

It's slower than using something like storage spaces/windows, but ya know...
http://www.infused.co.nz - My Blog.
infused
Enthusiast
 
Posts: 72
Liked: 3 times
Joined: Sat Apr 20, 2013 9:25 am
Full Name: Hayden Kirk

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby evander » Thu Mar 30, 2017 6:57 am

cbc-tgschultz wrote:
Different ransomware strains do things differently. Some will create a new copy of a file during encryption and then delete the old one when completed, and these might be stopped by blocking file extensions, but others will encrypt in-place and rename after the fact. In case of those, all you're doing is preventing the file from being renamed, making it hard to identify which files were hit; the data is still encrypted.

If you're doing FSRM, make sure you're taking advantage of its ability to send alerts. I personally use it to run scripts as well as alerting me. That way, under the right conditions, it will take automated action to limit the damage.

I totally agree, however I'm still optimistic that even if a particular ransomware strain does in-place encryption that it still has to make use of a tmp file of some sort to complete the process, in which case that file will be blocked if its in the same location. Also if it users the process memory to hold the tmp file it might still fail because the size of vbk files are generally much bigger than the available memory, right?
I confess I haven't done any testing to substantiate this thinking but I'd love to know if I'm right or wrong.
Additionally, if it makes a copy of the file, encrypts the copy (again with a possible tmp file of some sort) and then deletes the original (un-encrypted) file, chances of recovering the deleted files is a lot easier that recovering the encrypted file. The thing to take away is that by trying to protect just your backup file(s) versus trying to protect every single file in your organisation should be a lot easier and where I think its worth focusing on.

Overall I'm loving this thread and the different ideas that are being put forward.
evander
Enthusiast
 
Posts: 54
Liked: 3 times
Joined: Thu Nov 17, 2011 7:55 am

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby AIM_joshuag » Thu Mar 30, 2017 5:04 pm

MOBO wrote:how about if the repository is a standalone server and veeam program is the only system that has the credentials , am i still at risk for malware to delete backups ?

This would work only if the infection didn't infect a machine that you use those credentials from. If you log into veeam with the veeam credentials then a keylogger could use those to bring harm. If you really wanted to be sure you could have an offline laptop around that was used to check your backups from time to time.
AIM_joshuag
Novice
 
Posts: 3
Liked: never
Joined: Thu May 07, 2015 6:09 pm
Full Name: Joshua Garrett

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Fri Mar 31, 2017 3:32 pm 1 person likes this post

Mike Resseler wrote:The authentication details are stored in the DB and certainly not easily readable. That data in the DB is encrypted and you cannot just decrypt that. But that being said, a very specific written Trojan might succeed in getting that information out and then access the NAS and encrypt your backup files. Far fetched? Maybe, possible, unfortunately.

I'm afraid I must disagree that this is far fetched. As you said, the credentials are stored in the DB, and as much as you may obfuscate it the fact remains that the Veeam application must have the decryption key in order to decrypt the credentials. Therefore there is no cryptographic integrity to the credentials if an attacker has complete control of the host Veeam resides on. People unwind these kinds of schemes all the time, often just for fun and/or some prestige, it would be a mistake to believe that there isn't a tool already floating around the criminal underground that will accomplish this for an attacker.

Is it unlikely that you'll encounter an attacker that A) gets onto your network B) compromises your Veeam server, and C) has access to such a tool or the knowledge to do it himself? Maybe. But do you want to take that chance if you have another option?
cbc-tgschultz
Enthusiast
 
Posts: 40
Liked: 8 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby Mike Resseler » Mon Apr 03, 2017 8:07 am

Hi Tanner,

As I said, impossible... Unfortunately not :-(. But I do agree certainly that you do not want to take that change and have as much defense systems or layers (and don't forget the air-gap) as possible as it is getting worse and worse.
Mike Resseler
Veeam Software
 
Posts: 2638
Liked: 315 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby dellock6 » Mon Apr 03, 2017 10:05 pm 1 person likes this post

Actually you don't need this tool at all. Let's not forget that the goal here is not to steal data, it's to steal money. So I don't even need probably to open a Veeam console, I may target directly Veeam repositories to delete backup copies, so that the encrypted files are the only copies, so to force the victim to pay. If you read that thread on Reddit, the attacker encrypted the files, but then they just formatted veeam repos, they didn't need to encrypt those backups or anything else. Air gapping is the only real defense here.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 4833
Liked: 1263 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby y1008946 » Wed Apr 05, 2017 11:11 am

Hi, we have a NetApp 2552 where the data is stored and NetApp snapshots are taken.

We can see the NetApp snapshots in Veeam and can restore from them.

Are the NetApp snapshots protected or are they at risk too?

Thanks
y1008946
Enthusiast
 
Posts: 84
Liked: never
Joined: Mon Sep 23, 2013 3:56 pm

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Wed Apr 05, 2017 6:39 pm

dellock6 wrote:Actually you don't need this tool at all. Let's not forget that the goal here is not to steal data, it's to steal money. So I don't even need probably to open a Veeam console, I may target directly Veeam repositories to delete backup copies, so that the encrypted files are the only copies, so to force the victim to pay. If you read that thread on Reddit, the attacker encrypted the files, but then they just formatted veeam repos, they didn't need to encrypt those backups or anything else. Air gapping is the only real defense here.



It isn't the encryption that we're talking about protecting, but the credentials used to access the storage, which happen to be encrypted in the Veeam Database. Since Veeam needs to decrypt those credentials in order to use them, it must also store the decryption key somewhere, so my point was that a suitably savvy attacker could get at them anyway.

Of course, if the attacker has control of the Veeam server, they can much more simply direct Veeam to just delete all the backups anyway.

y1008946 wrote:Are the NetApp snapshots protected or are they at risk too?


If the attacker can get control of any system that has access to those snapshots and permission to delete them, then they are not safe.
cbc-tgschultz
Enthusiast
 
Posts: 40
Liked: 8 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby y1008946 » Thu Apr 06, 2017 10:28 am

Ok thanks,

The Netapp snapshots are meant to be read only, but I guess Veeam has a facility to delete them through the console?

It is the Netapp controlling and running the snapshots, the link with veeam is just so that we can see them.

I guess if we remove the storage from the veeam console and the credentials that should help.

Nearly all of the time if we need to restore, we do it from nightly backups stored on the Veeam server.

It would only be if we were desperate would we use storage snapshots, and at that point we could re add it to the console?

Thanks
y1008946
Enthusiast
 
Posts: 84
Liked: never
Joined: Mon Sep 23, 2013 3:56 pm

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Mon Apr 10, 2017 1:26 pm

I am unfamiliar with NetApp, let alone your configuration, but here's what you have to consider:

If Veeam can delete it, it should be considered vulnerable. If you can delete it using credentials that it would be possible to obtain via keylogging or by otherwise knowing where to find them, it should be considered vulnerable.

To be as safe as you can be under the conditions we've been discussing, you need at least one backup location requires physical access to destroy. If your NetApp snapshots can only be deleted by logging into NetApp from a physical terminal of some kind, or from a system that cannot be remoted into, then they should be ok.
cbc-tgschultz
Enthusiast
 
Posts: 40
Liked: 8 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby dellock6 » Mon Apr 10, 2017 7:54 pm

Daniel, what we are saying here (and I totally agree with Tanner) is that any system connected to the network can be targeted, yes you can make snapshots read-only, but if I get access to the netapp console, I can probably just delete them. What can help here are two things, but not accessible directly from the network by the attacker:
- 2FA (two-factor authentication) so that user needs to input a piece of credential that the attacker cannot get. Think like Google authenticator. There are solutions to implement it in a Windows RDP for example. It can be compromised if the attacker also gets access to your smartphone, but you surely lower the chances.
- the only real solution, as we are saying, it's air-gapping your backups. I write my backups to a tape, and I physically remove the tape from the library, and I even lock the mechanical read-only mechanism. The only way to destroy my backups at this point becomes to get physical access to those tapes as Tanner said, but this is really dangerous for the attacker, that usually prefers to hide himself behind the internet.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 4833
Liked: 1263 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby rmitura » Thu Apr 13, 2017 10:37 am

hello,
In my opinion:
1) if any windows host can write to CIFS shares than it can overwrite files with 0 length file. Altering file systems permissions would not help.
2) don't know if malware is capable of exploiting veeam harware snapshots API but if not netapp created snapshot should be safe.
3) in the old days I worked on openvms which had filing system with visioning (any file modification resulted in creating new one with higher version No). Guessing it would be easy to do crawler script that stripped permission from previous versions to local system user and run it on 10-20min scheduler. Not sure if there are file system versioning are still available but could be an option.
4) if malware only targets veeam files altering file extension instantly after backup job has finished (easy enough to add little script to your backup job) should be enough to protect them (won't work if malware encrypts everything if finds).
5) whitelisting application execution seams like good way forward (most likely it will need to be done on all veeam agent servers assuming no other device can contact your backup storage). It will make your system start, windows and antivirus update nightmare but if you do it right you wouldn't even need antivirus on your server.
6) detached volumes. Backup destination volumes are attached just before backup starts and detached after backup finish. Probably false sense of security (especially if your full backup takes 2 days) but better that than nothing.

If you think of anything else please share your ideas :-)
R. Mitura
rmitura
Lurker
 
Posts: 1
Liked: never
Joined: Tue Jan 31, 2017 10:47 am
Full Name: R Mitura

Previous

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: UNHStorage and 68 guests