Comprehensive data protection for all workloads
nebojsa
Service Provider
Posts: 20
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Veeam and EMC Data Domain Retention Lock

Post by nebojsa »

Hi,

has anyone tried integrating Veeam with EMC Data Domain with Retention Lock enabled Mtree as a repository?

I'm looking into storing weekly backups that need to be kept for 5 years on DD2200 (both CIFS and DD Boost are acceptable) and from what I understood, the retention period on a file needs to be set from the client side by modifying file's atime. I guess this could be done with a post-backup script and I'm just wondering if anyone's using Veeam in a similar scenario.

Thanks.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

Haven't you considered using backup copy jobs instead, for meeting your GFS retention requirements?
nebojsa
Service Provider
Posts: 20
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by nebojsa »

I haven't, because it seems that Backup Copy jobs allow a maximum of 99 weekly restore points, which is less than the required retention period.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

As far as I can get, when data is locked on an MTree, it cannot be overwritten or modified during the defined retention period. Unless there's an ability to lock files based on extension (so you could lock VBK files only), our jobs will have issues with updating VBM file (metadata). To prevent that, you need to set minimum retention period to something higher than the period of time between job cycles (however this needs to be tested).

As a workaround, if the requirement is to have backups on such a storage, you can copy them there from a regular repository using file copy job/some script or use VeeamZIP to send them there (also could be scripted).

Alternatively, you could use a regular backup job with weekly fulls that runs on a weekly schedule with retention of 260 restore points, unless using DD Retention Lock is required due to some compliance reasons.
nebojsa
Service Provider
Posts: 20
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by nebojsa »

Yeah, Retention Lock is there purely for compliance purposes.

My understanding is that setting a retention period on an Mtree isn't enough, you need to set the min/max retention on a per-file basis by modifying file's atime (so .vbm should be OK since I won't set any retention on it). My idea was also to do weekly full backups with 260 restore points with a post-backup script which sets the appropriate atime/retention on the created .vbk file.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

I'd check with EMC, however my understanding is that modifying file's atime is required to immediately lock the particular file, while without doing that the file is locked once it's modification time reaches the specified minimum retention period.
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

Hi,

Has anyone put this to production? we are looking into it, not for compliance but as an extra precaution against deletion of the backups.
We would like to apply this on normal backups and backup copy jobs. The lock does not have to be active within a few hours, but within a day.

I'm looking for some guidelines or tips in general and on how Veeam writes/reads the files etc. during backup.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

It depends on the backup method you're using (forward/forever forward/reverse incremental). In case of simple forward incremental, for example, files that are already written to disk, are never touched again (except metadata file), so you should be able to use it along with Retention Lock. How are you going to implement retention of older backups in this case?
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

Sorry for the late reply (i was on vacation).
We have the following methods in use:
Normal Backup:
  • Forward incrementals(daily)
  • Synthetic full backups every 7 days.
  • 28 restore points
  • Health check: every month last friday
BCJ:
  1. Copy every 14 days
  2. restorepoints to keep: 2
  3. Weekly: 4
  4. Monthly: 2
  5. Quarterly: 3
  6. yearly: 7
  7. Synthesized from incrementals
  8. health check every 2 months on last saterday
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

So....i've been doing some reading about the retention locks.
The Veeam files are saved via DD Boost. We do not use CIFS/NFS etc.
As of DD OS version 6 it is possible to set the locks on files via DD Boost.

i've gathered some information below
DD Boost is supported with both DD Retention Lock Governance and Compliance.
If client-side scripts are used to retention-lock backup files or backup images, and
if a backup application (Symantec NetBackup, for example) is also used on the
system via DD Boost, be aware that the backup application may not share the
context of the client-side scripts. Thus, when a backup application attempts to
expire or delete files that were retention locked via the client-side scripts, space is
not released on the EMC Data Domain system.

Data Domain recommends that administrators change their retention period policy
to align with the retention lock time. This applies to all the backup applications that
are integrated with DD Boost: Avamar, Symantec NetBackup, Symantec Backup
Exec, EMC NetWorker, and so on.
The Retention Period field indicates minimum and maximum retention
periods for the MTree. The retention period that is specified for a file in
the MTree must be equal to or greater than the minimum retention period
and equal to or less than the maximum retention period.
Retention lock functionality is available in two different flavours:
Governance: The less strict of the two retention lock flavours (i.e. locks against files can be reverted if necessary)
Compliance: The stricter of the two flavours which adheres to a number of common regulatory standards (i.e. locks against files cannot be reverted, the DDR must be configured with a 'security officer' user who must authenticate certain commands, and there are various restrictions on other functionality to prevent locked data from being removed/locks being reverted early)
Note that:

When retention lock is enabled against an mtree existing files within the mtree are *not* automatically locked (i.e. all pre-existing files remain read/write)
When a new file is written to an mtree with retention lock enabled the file is *not* automatically retention locked (i.e. the new file will remain read/write)
To retention lock a specific file the atime of the file must be modified to match the date/time until which the file should be retention locked (i.e. the date/time until which it should remain read only). Until the atime is modified in this way the file will *not* be retention locked (and can be modified/removed).
The steps are (I think):
  1. Enable CIFS
  2. Ensure that you have the retention lock license
  3. Enable DD retention lock on Mtree
  4. Use the touch command to lock files ( (sidenote: A files atime can be changed from an NFS/CIFS client using the 'touch' command)
Example script for setting the date after a job:
#Set the directory root for the script to run.
$dirlook=”P:\”
#This is setting the script to only check files with a modified date within the last 20 hours
$backdate=$(Get-Date).AddHours(-20)
#This is the number of days to set the access date to. Currently 21 days.
$forwarddate=$(Get-Date).AddDays(+21).ToString(‘MMddHHmmyyyy’)
#Find the Veeam Full Backup and Veeam Incremental Backup files that are modified in the last 20 hours.
Get-Childitem $dirlook -Recurse | `
where-object {!($_.psiscontainer)} | `
where { $_.LastWriteTime -gt $backdate -and ($_.Extension -eq ".vbk" -or $_.Extension -eq ".vib")} | `
foreach {C:\touch.exe -a -t $forwarddate $_.fullname}
Touch.exe download location: http://sourceforge.net/projects/unxutil ... p_redirect


Side note: Make shure the used DataDomain accounts do not have the same passwords as other accounts. The unit can stil be formatted via a re init.
Enabling DD Retention Lock Compliance enforces many restrictions on lowlevel
access to system functions used during troubleshooting. Once enabled,
the only way to disable DD Retention Lock Compliance is to initialize and reload
the system, which results in destroying all data on the system.

I am unaware if its posible to accomplish this via DD boost only. EMC states that Veeam supports the retention lock. But both EMC and Veeam do not have any Veeam/EMC combo guide.
Image

Are there any plans to integrate this feature in the near future?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

Current behavior doesn't depend on the repository type - it works similarly for CIFS and DD Boost repositories. If you set Retention Lock in a way that it releases the file on DD by the moment Veeam B&R wants to delete it according to it's own retention, there should not be any issues.
adb98
Enthusiast
Posts: 63
Liked: 13 times
Joined: Jul 21, 2016 5:03 pm
Full Name: Aaron B
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by adb98 » 1 person likes this post

Why don't you just run with a GFS for each year. I know that is a pita but it does allow for easier search and review. Each year create a new GFS and delete the old job but keep the data. You will see it in imported. If you really wanted to get cheeky with it, you could create separate Mtrees though there is a limit on how many Mtrees you can have and I don't know how many you are currently using. Hopefully more that just one for Veeam. :D

Just my two sense.
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

Our goal is that the files cannot be deleted by and unwanted person. And we want to achive this goal automated.
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

Foggy, are there any plans to implement the retention lock feature in Veeam?

It is possible to set locks via DDboost. It would be great to have an extra field in the job to enable retetion lock and how many days to be locked. Keeping in mind that the user also has to have retention lock enabled on data domain.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

Do you mean configuring DD Retention Lock via Veeam B&R UI?
martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah »

Yes, thats correct. Then we can manage this within Veeam instead of creating powershell scripts. Also we have to enable CIFS just to activate the retention lock and we want to avoid using CIFS.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » 1 person likes this post

Thanks for the details. More tight integration with the Retention Lock feature is on our list.
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by namiko78 »

Foggy, is there any update to this feature? I too would like to utilize retention lock with our new data domain.
Thank you
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by veremin »

Nothing we can provide you with at the moment. Thanks!
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by namiko78 »

Just wondering about an ETA, Is it still on the list?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by veremin »

As mentioned, we cannot share any details or ETA right now. Thank you for understanding.
adb98
Enthusiast
Posts: 63
Liked: 13 times
Joined: Jul 21, 2016 5:03 pm
Full Name: Aaron B
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by adb98 »

Data Domain Retention locks would be nice. Maybe have it like it is set for tapes.

The way we currently do it as follows.

1. Create an Mtree with the year you are wanting to save. Do all your saves until its ready to be archived.

2. Delete the said jobs....This will move it to disk (imported).

3. Go into disk (imported) and remove from catalog. This removes it from being seen in Veeam.

3. Remove the repository (mtree) from Veeam

Now Veeam won't see it and doesn't have access to the repository. If you need it again... Add the repository and scan for backups. It will bring them all back into disk (imported). If you have someone who can remove files directly from the DD then you have a bigger issue. Locking a file with someone who has access to the DD is not going to protect it.
Trelor
Enthusiast
Posts: 47
Liked: 15 times
Joined: Apr 27, 2015 6:02 pm
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by Trelor »

foggy wrote: Jan 25, 2018 2:56 pm Thanks for the details. More tight integration with the Retention Lock feature is on our list.
I assume that this is still on the list of features? We just bought 2 data domains and I am interested in this feature as well.
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by namiko78 »

For some reason, they can't even confirm it's still on the list. Not sure why it's so top secret? :)
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by Gostev » 2 people like this post

Hello. This feature is not in a short-term roadmap. Thanks!
kaithost
Veeam ProPartner
Posts: 26
Liked: 1 time
Joined: Aug 25, 2011 7:53 am
Full Name: Kai Thost
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by kaithost »

Hi, since with V10 there is S3 object locking (at least with Amazon S3 as far a I know) i'd like to push this again.
Since DD retention locking should be pretty easy to implement and due to the high risk of data loss by malware/ransomware
I'd think it should be pretty easy to add retetion locking within the DD to the short-term roadmap, eh?

DellEMC hasn't have to change anything, the command set is easily utilized by DDboost, so what is the problem?
Your customers will love you for that!
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

Hi Kai, thank you for the feedback. S3 object locking is another part of Veeam B&R though and so far the number of requests is not that large for this feature to get prioritized.
kaithost
Veeam ProPartner
Posts: 26
Liked: 1 time
Joined: Aug 25, 2011 7:53 am
Full Name: Kai Thost
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by kaithost »

Hi Foggy,

thank you for the quick reply.

Over all and knowing about the ease of use with retention locking on a DD this should be an easy win for Veeam to implement.
Sounds like a value feature for an Enterprise Plus license and a additional motivation to get customers to Eplus licenses.

Our customers achieve great results with DD backends as a last line of defense and even with ransomware protection by DDs.
It's just so much nicer with retention locking put by Veeam ;)

So i'd guess most customers with DDs are simply not using the forum and the demand would be much higher than you think :)

Best regards
Kai
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy »

Forum is not the only source of feedback we get, but your considerations are taken, thanks anyway!
SE-1
Influencer
Posts: 22
Liked: 5 times
Joined: Apr 07, 2015 1:42 pm
Full Name: Dirk Slechten
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by SE-1 » 3 people like this post

Hello Foggy & Gostev,

We are an IT integrator, and we receive a lot of questions lately of our customers regarding this.
We and most of our customers are using veeam in combination with Data Domain, and all are very pleased about this.
Having the retention lock "checkbox" on DDBOOST jobs is something we would directly implement when this feature would be available.

Kind Regards
Dirk
Post Reply

Who is online

Users browsing this forum: Gostev and 145 guests