Comprehensive data protection for all workloads
Post Reply
pmichelli
Expert
Posts: 115
Liked: 31 times
Joined: Mar 16, 2023 5:47 pm
Contact:

Data Domain Replication / Fast Copy Script

Post by pmichelli » 3 people like this post

Mods : If this is not appropriate, please feel free to remove the post. I figure this would be useful to some individuals.

Config You have 2 Data Domain that replicate to each other using native replication from the appliances. One is mounted to Veeam and writes the active backups. The other is not mounted to Veeam

Scenario Native replication takes place every 15 min. Say you have a mTREE called VEEAM where the backups write to and it replicates to the second appliance to a folder example VEEAM-DR. If anything happens to the main mTREE on the one Veeam can see (data is corrupted with malware, someone erases the repository data, etc) those changes will replicate to the second appliance and you lose all your data. This scenario assumes you are not using immutability on the Data Domain(s). So how can you work around this?

You can create a new mTREE and Fast Copy on the second appliance. In the event the primary is wiped and it replicates those changes and erases the same data on the second side, you can make the Fast Copy (aka snapshot) read / write and mount it to Veeam. This is a manual process..... So I wrote a script to automate all this and I wanted to share it to the Veeam community.

Script Function
You need to be using password less SSH keys to access your Data Domain.
The script will connect to the target Data Domain and obtain a list of mTREE. It will parse the output and grabs only the mTREE name. It excludes any mTREE marked with a D (deleted)
It then creates a new mTREE with the same name and appends a _YYYYMMDD to it and then performs a Fast Copy
The script ignores any mTREE that already has a _YYYYMMDD appended (so you don't end up taking Fast Copies of Fast Copies)
The script keeps 3 days worth of mTREE Fast Copies and deletes any that are older.
The script sends an email when completed (I use msmtp but sendmail or anything else that sends mail will work)
I run the script via a cron job
If you need to do DR testing, you can make any of the Fast Copies read/write (I can provide steps if necessary). If you turn the RO replication mTREE into RW for testing, it breaks the replication, so this is another bonus that you can validate the replicated data without breaking replication.

Code: Select all

#!/bin/bash

# Define variables
DD_HOST="your-datadomain-host"
DD_USER="your-username"
DATE=$(date +%Y%m%d)  # Format: YYYYMMDD
DATE_3_DAYS_AGO=$(date -d "-3 days" +%Y%m%d)  # Date 3 days ago (for comparison)
SSH_KEY_PATH="$HOME/.ssh/id_rsa"  # Path to your private SSH key (default location)

# Function to run commands on the Dell Data Domain using SSH key-based authentication
run_dd_command() {
    ssh -i "$SSH_KEY_PATH" -o StrictHostKeyChecking=no "$DD_USER@$DD_HOST" "$1"
}

# Step 1: List all MTrees with their status, filter only the MTree names without "/data/col1/" prefix, and exclude MTrees with status 'D'
MTREES=$(run_dd_command "mtree list" | awk '/^\/data\/col1/ && $3 != "D" {name=$1; sub("^/data/col1/", "", name); print name}')

# Step 2: Check and delete MTrees older than 3 days with a _YYYYMMDD suffix
for MTree in $MTREES; do
    # Check if the MTree ends with _YYYYMMDD
    if [[ "$MTree" =~ _([0-9]{8})$ ]]; then
        # Extract the date (YYYYMMDD) from the MTree name
        MTreeDate="${BASH_REMATCH[1]}"
        
        # Compare the extracted date with the date 3 days ago
        if [[ "$MTreeDate" < "$DATE_3_DAYS_AGO" ]]; then
            # Delete the MTree if it's older than 3 days
            echo "Deleting MTree $MTree because it is older than 3 days."
            run_dd_command "mtree delete /data/col1/$MTree"
        fi
    fi
done

# Step 3: Create new MTrees and perform fast copy for each MTree that does not end with a 4 or 8 digit date
for MTree in $MTREES; do
    # Check if the MTree ends with a 4-digit or 8-digit number after an underscore (e.g., _1105 or _20231111)
    if [[ "$MTree" =~ _[0-9]{4}$ || "$MTree" =~ _[0-9]{8}$ ]]; then
        echo "Skipping MTree $MTree because it ends with a 4-digit or 8-digit date."
        continue
    fi

    # Define new MTree name with the current date appended
    NEW_MTree="${MTree}_$DATE"
    NEW_MTree_Path="/data/col1/$NEW_MTree"
    
    # Step 3.1: Create the new MTree
    echo "Creating MTree: $NEW_MTree_Path"
    run_dd_command "mtree create $NEW_MTree_Path"

    # Step 3.2: Perform fast copy from the original MTree to the new MTree
    echo "Performing fast copy from $MTree to $NEW_MTree_Path"
    run_dd_command "filesys fastcopy force source /data/col1/$MTree destination $NEW_MTree_Path"
    
    echo "Fast copy from $MTree to $NEW_MTree_Path completed"
done

echo "Script completed. All changes have been applied."

# Send email notification using msmtp
SUBJECT="Dell Data Domain Script Completed"
EMAIL_BODY="The Dell Data Domain script has completed its tasks. All MTrees were checked, old ones deleted, and new ones created with fast copy operations."

echo -e "Subject:$SUBJECT\n\n$EMAIL_BODY" | msmtp recipient-email@domain.com
Here is an example of what the folders look like on the Data Domain

Code: Select all

Welcome to Data Domain OS 8.0.0.20-1107705
------------------------------------------
automate@datadomain#
automate@datadomain# mtree list
Name                                        Pre-Comp (GiB)   Status
-----------------------------------------   --------------   ------
/data/col1/Archive                                        		9293.8   RW
/data/col1/Archive_20241130                         		9293.8   RW
/data/col1/Archive_20241201                         		9293.8   RW
/data/col1/Archive_20241202                         		9293.8   RW
/data/col1/Archive_20241203                         		9293.8   RW
/data/col1/VEEAM-UO-CORE-DR                               37295.9   RO/RD
/data/col1/VEEAM-UO-CORE-DR_20241130               35843.2   RW
/data/col1/VEEAM-UO-CORE-DR_20241201               37295.9   RW
/data/col1/VEEAM-UO-CORE-DR_20241202               37295.9   RW
/data/col1/VEEAM-UO-CORE-DR_20241203               37295.9   RW
Mildur
Product Manager
Posts: 10277
Liked: 2746 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Data Domain Replication / Fast Copy Script

Post by Mildur »

Hello pmichelli

Thank you for sharing.

Best,
Fabian
Product Management Analyst @ Veeam Software
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Semrush [Bot] and 128 guests