Availability for the Always-On Enterprise
nebojsa
Service Provider
Posts: 14
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Veeam and EMC Data Domain Retention Lock

Post by nebojsa » Feb 06, 2016 4:27 pm

Hi,

has anyone tried integrating Veeam with EMC Data Domain with Retention Lock enabled Mtree as a repository?

I'm looking into storing weekly backups that need to be kept for 5 years on DD2200 (both CIFS and DD Boost are acceptable) and from what I understood, the retention period on a file needs to be set from the client side by modifying file's atime. I guess this could be done with a post-backup script and I'm just wondering if anyone's using Veeam in a similar scenario.

Thanks.

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Feb 07, 2016 3:46 pm

Haven't you considered using backup copy jobs instead, for meeting your GFS retention requirements?

nebojsa
Service Provider
Posts: 14
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by nebojsa » Feb 09, 2016 8:25 am

I haven't, because it seems that Backup Copy jobs allow a maximum of 99 weekly restore points, which is less than the required retention period.

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Feb 09, 2016 1:04 pm

As far as I can get, when data is locked on an MTree, it cannot be overwritten or modified during the defined retention period. Unless there's an ability to lock files based on extension (so you could lock VBK files only), our jobs will have issues with updating VBM file (metadata). To prevent that, you need to set minimum retention period to something higher than the period of time between job cycles (however this needs to be tested).

As a workaround, if the requirement is to have backups on such a storage, you can copy them there from a regular repository using file copy job/some script or use VeeamZIP to send them there (also could be scripted).

Alternatively, you could use a regular backup job with weekly fulls that runs on a weekly schedule with retention of 260 restore points, unless using DD Retention Lock is required due to some compliance reasons.

nebojsa
Service Provider
Posts: 14
Liked: 2 times
Joined: Nov 15, 2010 11:19 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by nebojsa » Feb 09, 2016 3:51 pm

Yeah, Retention Lock is there purely for compliance purposes.

My understanding is that setting a retention period on an Mtree isn't enough, you need to set the min/max retention on a per-file basis by modifying file's atime (so .vbm should be OK since I won't set any retention on it). My idea was also to do weekly full backups with 260 restore points with a post-backup script which sets the appropriate atime/retention on the created .vbk file.

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Feb 09, 2016 4:49 pm

I'd check with EMC, however my understanding is that modifying file's atime is required to immediately lock the particular file, while without doing that the file is locked once it's modification time reaches the specified minimum retention period.

martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah » Dec 22, 2017 2:57 pm

Hi,

Has anyone put this to production? we are looking into it, not for compliance but as an extra precaution against deletion of the backups.
We would like to apply this on normal backups and backup copy jobs. The lock does not have to be active within a few hours, but within a day.

I'm looking for some guidelines or tips in general and on how Veeam writes/reads the files etc. during backup.

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Dec 22, 2017 4:40 pm

It depends on the backup method you're using (forward/forever forward/reverse incremental). In case of simple forward incremental, for example, files that are already written to disk, are never touched again (except metadata file), so you should be able to use it along with Retention Lock. How are you going to implement retention of older backups in this case?

martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah » Jan 02, 2018 7:52 am

Sorry for the late reply (i was on vacation).
We have the following methods in use:
Normal Backup:
  • Forward incrementals(daily)
  • Synthetic full backups every 7 days.
  • 28 restore points
  • Health check: every month last friday
BCJ:
  1. Copy every 14 days
  2. restorepoints to keep: 2
  3. Weekly: 4
  4. Monthly: 2
  5. Quarterly: 3
  6. yearly: 7
  7. Synthesized from incrementals
  8. health check every 2 months on last saterday

martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah » Jan 08, 2018 9:38 am

So....i've been doing some reading about the retention locks.
The Veeam files are saved via DD Boost. We do not use CIFS/NFS etc.
As of DD OS version 6 it is possible to set the locks on files via DD Boost.

i've gathered some information below
DD Boost is supported with both DD Retention Lock Governance and Compliance.
If client-side scripts are used to retention-lock backup files or backup images, and
if a backup application (Symantec NetBackup, for example) is also used on the
system via DD Boost, be aware that the backup application may not share the
context of the client-side scripts. Thus, when a backup application attempts to
expire or delete files that were retention locked via the client-side scripts, space is
not released on the EMC Data Domain system.

Data Domain recommends that administrators change their retention period policy
to align with the retention lock time. This applies to all the backup applications that
are integrated with DD Boost: Avamar, Symantec NetBackup, Symantec Backup
Exec, EMC NetWorker, and so on.
The Retention Period field indicates minimum and maximum retention
periods for the MTree. The retention period that is specified for a file in
the MTree must be equal to or greater than the minimum retention period
and equal to or less than the maximum retention period.
Retention lock functionality is available in two different flavours:
Governance: The less strict of the two retention lock flavours (i.e. locks against files can be reverted if necessary)
Compliance: The stricter of the two flavours which adheres to a number of common regulatory standards (i.e. locks against files cannot be reverted, the DDR must be configured with a 'security officer' user who must authenticate certain commands, and there are various restrictions on other functionality to prevent locked data from being removed/locks being reverted early)
Note that:

When retention lock is enabled against an mtree existing files within the mtree are *not* automatically locked (i.e. all pre-existing files remain read/write)
When a new file is written to an mtree with retention lock enabled the file is *not* automatically retention locked (i.e. the new file will remain read/write)
To retention lock a specific file the atime of the file must be modified to match the date/time until which the file should be retention locked (i.e. the date/time until which it should remain read only). Until the atime is modified in this way the file will *not* be retention locked (and can be modified/removed).
The steps are (I think):
  1. Enable CIFS
  2. Ensure that you have the retention lock license
  3. Enable DD retention lock on Mtree
  4. Use the touch command to lock files ( (sidenote: A files atime can be changed from an NFS/CIFS client using the 'touch' command)
Example script for setting the date after a job:
#Set the directory root for the script to run.
$dirlook=”P:\”
#This is setting the script to only check files with a modified date within the last 20 hours
$backdate=$(Get-Date).AddHours(-20)
#This is the number of days to set the access date to. Currently 21 days.
$forwarddate=$(Get-Date).AddDays(+21).ToString(‘MMddHHmmyyyy’)
#Find the Veeam Full Backup and Veeam Incremental Backup files that are modified in the last 20 hours.
Get-Childitem $dirlook -Recurse | `
where-object {!($_.psiscontainer)} | `
where { $_.LastWriteTime -gt $backdate -and ($_.Extension -eq ".vbk" -or $_.Extension -eq ".vib")} | `
foreach {C:\touch.exe -a -t $forwarddate $_.fullname}
Touch.exe download location: http://sourceforge.net/projects/unxutil ... p_redirect


Side note: Make shure the used DataDomain accounts do not have the same passwords as other accounts. The unit can stil be formatted via a re init.
Enabling DD Retention Lock Compliance enforces many restrictions on lowlevel
access to system functions used during troubleshooting. Once enabled,
the only way to disable DD Retention Lock Compliance is to initialize and reload
the system, which results in destroying all data on the system.

I am unaware if its posible to accomplish this via DD boost only. EMC states that Veeam supports the retention lock. But both EMC and Veeam do not have any Veeam/EMC combo guide.
Image

Are there any plans to integrate this feature in the near future?

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Jan 08, 2018 12:32 pm

Current behavior doesn't depend on the repository type - it works similarly for CIFS and DD Boost repositories. If you set Retention Lock in a way that it releases the file on DD by the moment Veeam B&R wants to delete it according to it's own retention, there should not be any issues.

adb98
Enthusiast
Posts: 39
Liked: 4 times
Joined: Jul 21, 2016 5:03 pm
Full Name: Aaron B
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by adb98 » Jan 08, 2018 2:31 pm 1 person likes this post

Why don't you just run with a GFS for each year. I know that is a pita but it does allow for easier search and review. Each year create a new GFS and delete the old job but keep the data. You will see it in imported. If you really wanted to get cheeky with it, you could create separate Mtrees though there is a limit on how many Mtrees you can have and I don't know how many you are currently using. Hopefully more that just one for Veeam. :D

Just my two sense.

martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah » Jan 08, 2018 2:59 pm

Our goal is that the files cannot be deleted by and unwanted person. And we want to achive this goal automated.

martjah
Service Provider
Posts: 6
Liked: never
Joined: Apr 07, 2014 7:09 am
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by martjah » Jan 17, 2018 2:00 pm

Foggy, are there any plans to implement the retention lock feature in Veeam?

It is possible to set locks via DDboost. It would be great to have an extra field in the job to enable retetion lock and how many days to be locked. Keeping in mind that the user also has to have retention lock enabled on data domain.

foggy
Veeam Software
Posts: 17097
Liked: 1397 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam and EMC Data Domain Retention Lock

Post by foggy » Jan 19, 2018 4:10 pm

Do you mean configuring DD Retention Lock via Veeam B&R UI?

Post Reply

Who is online

Users browsing this forum: tgietz and 46 guests