Yes, Ransomware can delete your Veeam backups.

Availability for the Always-On Enterprise

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby evander » Fri Mar 17, 2017 11:52 am

Thinking out loud a bit further - possibly a script that would change NTFS permissions on a file or folder that held VBKs pre and post Veeam backup job.
I'm assuming the ransomware would need at least write/modify NTFS permission on the file to encrypt it right? So if 90% of the time that file/folder has NTFS permissions restricted to "read only" then its another layer of protection. I understand the ransomware might be smart enough to change NTFS permissions itself if its running under an admin account but possibly that can be mitigated by some way I haven't thought of yet.

With the power of Veeam my entire organization can be backed up into a handful of files or everything under one top level folder, If we put our minds to it I'm sure we can find a way to protect that folder so that only Veeam can make changes to it, or at a minimum changes can only be made to it only during a small and specific time period.
evander
Enthusiast
 
Posts: 50
Liked: 2 times
Joined: Thu Nov 17, 2011 7:55 am

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby tacioandrade » Mon Mar 20, 2017 2:29 am

Great idea really, I had not thought of it then .... In case the problem would be only in the part of overwriting the old backup to remove the new one, however it really is a great idea to start working on.

Sincerely, Tácio Andrade.
tacioandrade
Enthusiast
 
Posts: 27
Liked: 3 times
Joined: Thu Nov 17, 2016 2:04 am
Full Name: Tácio Andrade

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby TrevorBell » Mon Mar 20, 2017 1:30 pm

Hi All,

Just to quick post that something similar happened to me at a previous company... ill share some facts what happened in that scenario.
Had a call one night a director couldn't get email. Then had call from vendor who supports 24/7 infrastructure saying he can see SQL servers being encrypted.
By the time I logged in via VPN 43 servers were hit and 110 use laptops / desktops.
SAMAS was the cause... Now lets look at how it happened..

RRAS server was still being used and was bruteforced as it was a weak password which allowed hackers in and that hacked AD to make an account called administrators. Hackers were in the system for approx. 6 days before to attack took place hence if you call your server VEEAM or VBK etc its easy to locate it - remember its easy for other to identify too. The RRAS server Ip was published on showdan.io and 4 days after it appeared on there as having an RRAS server it was hit.

I had offsite tape backups and recalled them from Iron Mountain and started restore process. 10TB of data wasn't too bad. But the board of the company took the decision it was easier to pay the £15,000 in bitcoins than wait for all laptops to recovered by IT so it was paid and the encryption keys obtained, yes not ideal but in a large PLC it was small change.

There are many ways to hide the server and use other techniques but security starts with every admin. Files don't just disappear with SAMAS they get deleted and then encryption to stop restores being possible ..

We actually caught the SAMAS strain and passed to Symantec and FBI as it wasn't known the strain back in October last year.
We also re-wrote the DEC2.EXE decrypt exe from 3 passes at decoding - it only needed 1 pass speeding the de-crypt process up ( not that the other 2 passes did anything except waste time )

Indeed SSL VPN was installed, yes user account and the lazy IT folks who used the same passwords for both user and their domain admins got educated.

People need to ( if you don't already ) pay for a pen test and see the amount of vulnerabilities it comes back with.. in this case it was over 6,000 redial actions from VMware exploits, windows update exploit, java exploits even Symantec AV Client exploits... IIS exploits SSL exploits even if you think your infrastructure is upto date its worth calling a pen tester in for a day or two... might be the best thing you do this year :)

Thanks

Trev..
TrevorBell
Expert
 
Posts: 329
Liked: 1 time
Joined: Fri Feb 13, 2009 10:13 am
Location: Redditch UK
Full Name: Trevor Bell

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Mon Mar 20, 2017 2:21 pm 2 people like this post

I disagree with the idea of paying for a Pen Test. Here's why: They will always find stuff. Always. And you'll fix it, and then you'll get another Pen Test a while later and they'll find more stuff. Always. And you'll fix it and pay for another Pen Test...

See, the problem is that the industry hasn't taken security seriously... basically ever. It will continue to not take security seriously for the foreseeable future. You don't have any control over most of the software you use in your business, but even if you did it is unlikely you'd ever find and patch all the vulnerabilities and new ones would be introduced with every new feature or update you do, as they are with third party software, because the industry does not take security seriously.

And that's largely because human beings do not take security seriously, because we're just naturally awful at risk assessment. But I digress.

Point being, it is much, much cheaper to, instead of paying for endless Pen Tests, just assume everything is vulnerable, always, and design your practices around that. Assume that someone will break into any and all public facing services, ok, now where can they go from there? Assume admin credentials will be stolen, what can we do about that? Assuming the worst happens, what's our plan for dealing with it? How could that be compromised? Etc. You can't buy security, because it is a mindset.

Ultimately, this particular threat is never going to go away. This is obvious from statements like "But the board of the company took the decision it was easier to pay the £15,000 in bitcoins than wait for all laptops to recovered by IT", so look forward to the bright future of IT where your budget includes K&R insurance. It's easier than security anyway.
cbc-tgschultz
Enthusiast
 
Posts: 37
Liked: 7 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby Tonksy » Thu Mar 23, 2017 7:21 am

Using Veeam with a NAS presenting CIFS, NFS, REFS etc exposes the shares. ExaGrid has Veeams own data mover software within appliances adding addtional protection from ransomware.

Insulation from Ransomware
When ransomware strikes, it is critical to have backups insulated from the malicious encryption/damage since they may be your last line of defense. ExaGrid helps insulate backups in the following ways:

1. Comprehensive access security ExaGrid shares can be accessed only from designated backup/media servers. While those severs may also be subject to rampant ransomware, the fewer servers that have access to your backups, the better.

2. SMB signing can be enabled for ExaGrid shares, requiring Windows account credentials to be authenticated and authorized before access is granted to an ExaGrid share, further reducing the chance of malicious access to backups.

3 Veeam Accelerated Data Mover shares require a separate Veeam password and are accessible only via SSH, which also reduces the chance of malicious access to Veeam backups.

4 All accounts used to manage the ExaGrid software are protected using non-default passwords. This includes thebackup “admin” account, the special ExaGrid customer support account, and root access.

5 ExaGrid software is updated at least quarterly with the latest appropriate CVE fixes, reducing the ways ransomware can gain access to ExaGrid servers. Software may be updated more frequently as dictated by CVE severity.

6 Each ExaGrid server runs a proper firewall and a customized Linux distribution that opens just the ports and runs just the services necessary for receiving backups, web-based GUI, and ExaGrid-to-ExaGrid replication.

4. Communications between ExaGrid servers is secured using Kerberos authorization and authentication, protecting from a“man in the middle” attack from malicious users or software.

Finally
Periodic Assessments Using Beyond Trust’s Retina Network Vulnerability ScannerA complete vulnerability assessment is run periodically against ExaGrid’s software using the Retina Network Vulnerability Scanner.This tool is the security industry’s most respected and validated vulnerability assessment tool. Audit risks identified by Retina are
evaluated by ExaGrid engineering and appropriate resolution is applied.
Tonksy
Lurker
 
Posts: 1
Liked: never
Joined: Thu Mar 23, 2017 7:08 am
Full Name: Mike Tonks

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby frankj » Thu Mar 23, 2017 11:08 am

Woudl this be a safe bet ?

Tape autoloader for our 7 TB prod store on veeam

24 tapes rounded in 8 sets of 3 rotated weekly.

giving us a buffer of 8 weeks of tape backup, with a possibility of 1 week old data to 8 weeks if all where corrupted but the last one.

Would an autoloader be safe for air gap ?
frankj
Service Provider
 
Posts: 20
Liked: 1 time
Joined: Fri May 27, 2016 4:53 pm
Full Name: FRANK Jacques

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby nitramd » Thu Mar 23, 2017 2:45 pm

With all of the good ideas that the community has proposed, has anyone factored in how RTO would be impacted?

For example, if one decides to rename file extensions of the backup files (pre/post script), how long would it take to recover a file/VM? This might also apply to an air gapped server.
nitramd
Influencer
 
Posts: 14
Liked: never
Joined: Thu Feb 16, 2017 8:05 pm

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Thu Mar 23, 2017 2:49 pm

Well of course having to restore from an air-gapped source will add to your RTO, but that's a worst-case scenario where your RTO would otherwise be infinity. If the company you work for is the kind that will pay the ransom rather than wait, then it may not be worth your effort.
cbc-tgschultz
Enthusiast
 
Posts: 37
Liked: 7 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby nitramd » Thu Mar 23, 2017 2:55 pm

I don't disagree with what you're saying. However, if you've gone on record saying that your RTO will be 10 minutes, or whatever value, and you have not updated that estimate/guarantee since the advent of ransomware then your you may have put yourself in a sling with management.
nitramd
Influencer
 
Posts: 14
Liked: never
Joined: Thu Feb 16, 2017 8:05 pm

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby yasuda » Thu Mar 23, 2017 3:51 pm

cbc-tgschultz wrote:Well of course having to restore from an air-gapped source will add to your RTO, but that's a worst-case scenario where your RTO would otherwise be infinity. If the company you work for is the kind that will pay the ransom rather than wait, then it may not be worth your effort.


Except paying the ransom is no guarantee you will be able to decrypt your data.
yasuda
Enthusiast
 
Posts: 42
Liked: 9 times
Joined: Thu May 15, 2014 3:29 pm
Full Name: Peter Yasuda

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby cbc-tgschultz » Thu Mar 23, 2017 4:07 pm

In general it is in the best interests of the ransoming parties to uphold their end of the deal, and I've only ever heard of scattered cases where that hasn't happened. It's a risk, to be sure, because you're dealing with unscrupulous individuals after all, but it isn't much of one.

Personally, I know that my company would only consider it as a last resort, so I just have to worry about how to get us to a point where we can survive the kind of worst-case scenario being described here, not how long it is going to take to recover or even so much what the RPO is. For others, well, we've already had someone comment that their company decided to pay the ransom to recover laptops rather than wait on them to be restored from the backup data they had.
cbc-tgschultz
Enthusiast
 
Posts: 37
Liked: 7 times
Joined: Fri May 13, 2016 1:48 pm
Full Name: Tanner Schultz

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby yasuda » Thu Mar 23, 2017 4:18 pm

frankj wrote:Woudl this be a safe bet ?
Tape autoloader for our 7 TB prod store on veeam
24 tapes rounded in 8 sets of 3 rotated weekly.
giving us a buffer of 8 weeks of tape backup, with a possibility of 1 week old data to 8 weeks if all where corrupted but the last one.
Would an autoloader be safe for air gap ?


Do the tapes stay in the autoloader? If so, then in the original poster's scenario, no. In that case, the attacker had remote admin access, and, assuming your autoloader is not managed by an air gapped server, it could be commanded to overwrite tapes with garbage.

If you could air gap the tape management server, and still get data to the tapes, that would be good. An interesting architectural challenge.

I think a cloud backup service like Carbonite or Crashplan would be very good for protecting your file data from ransomware, although not perfect. Barracuda Backup Server is probably the best single solution for ransomware protection plus image backup, because it replicates deduped backups off site, and it is managed through a cloud service so you never need to log onto it from the local network.

I don't think there is a really great solution for the scenario where an intruder has undetected access to your network for an extended period of time, because given that, you can imaging Mr Robot scenarios where nothing is safe. Really, your focus should be on putting systems and processes into place to give you a high probability of detecting the breach.

I also think it would be more productive to discuss methods of protecting your backups, where we assume automated ransomware (no active command and control) or that the breach will be detected before the attacker gains unlimited root access to everything. Physical locks are rated in terms of how long they will delay an attacker, and we should rate backup protection the same way. How long will it take a determined attacker to destroy your backups?
yasuda
Enthusiast
 
Posts: 42
Liked: 9 times
Joined: Thu May 15, 2014 3:29 pm
Full Name: Peter Yasuda

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby yasuda » Thu Mar 23, 2017 4:34 pm

cbc-tgschultz wrote:In general it is in the best interests of the ransoming parties to uphold their end of the deal, and I've only ever heard of scattered cases where that hasn't happened. It's a risk, to be sure, because you're dealing with unscrupulous individuals after all, but it isn't much of one.


Sure, but if you're not considering low probability events, why are you reading the thread?

I think I missed your original point, however, if it was that you need to do the cost to benefit analysis, and I completely agree with that.

And set priorities. If you're not confident a breach will be detected in less than the average 100+ days, address that first. And are you even testing your backups? If you're not running SureBackup tests, ransomware should be a lower priority. Do you test tapes when they rotate back from off site storage?
yasuda
Enthusiast
 
Posts: 42
Liked: 9 times
Joined: Thu May 15, 2014 3:29 pm
Full Name: Peter Yasuda

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby nitramd » Thu Mar 23, 2017 4:37 pm

yasuda wrote:Do the tapes stay in the autoloader?

Yes, at least the ones I've used.

yasuda wrote:I don't think there is a really great solution for the scenario where an intruder has undetected access to your network for an extended period of time...

Not completely true. I've heard from Justice Department officials that there are examples of companies who have been unaware of breaches for years; they did not give the number of companies in this category, they did not name names nor give the exact amount of time that elapsed before discovery.

Although, Yahoo does come to mind.
nitramd
Influencer
 
Posts: 14
Liked: never
Joined: Thu Feb 16, 2017 8:05 pm

Re: Yes, Ransomware can delete your Veeam backups.

Veeam Logoby yasuda » Thu Mar 23, 2017 4:44 pm

Random thought about sending tapes off site: If I'm in your network, maybe I've found email or documents, maybe I've found a copy of your DR plan in your wiki. So I call Iron Mountain and say, "We need all our backup tapes delivered to our DR site ASAP! Here's the address..." And for a reasonable fee, not only will I send you the key to decrypt your data, I will mail your tapes to you.

Plausible?
yasuda
Enthusiast
 
Posts: 42
Liked: 9 times
Joined: Thu May 15, 2014 3:29 pm
Full Name: Peter Yasuda

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: dellock6, MSNbot Media, tdewin and 62 guests