Backup to Tape from DeDup Appliance (DataDomain) results in slow speeds

[ninjaburn81]

So after researching the board, I believe I have the answer to this question, but I want it put in front of everyone first.

We are implementing tape for various reasons (mainly for air-gapping of data), and at the moment we have daily backups to tape running fine with minimal muss and fuss. However, when I started bring some Weekly/Monthly full backups into the mix, I noticed an issue. The length of time to complete was insane...40 HOURS for 2.4TB of data (unstructured/file shares). After researching, I have two culprits:

1) Encryption enabled: enabled since unlike the dailys, they are being removed from the locked enclosure.
2) The file shares, unlike most of the other backups, reside on a Data Domain box; thus, dedup is involved, which is probably increasing times.

Fixing #1 is easy but I'm hoping fixing #2 is enough. My best thought to fix this is to to fire off a copy job of the file share backups that copies the latest full backup chain to a non-dedup box (either a SAN or my JBOD Veeam backup server), THEN have my weekly/monthly job run off of that. Does everyone feel that is the best way to fix the hydration issue, or is there something else I could do that would be cleaner? I feel like having the double jump to get running is a bit messy, but I'm not sure what other options I have available in this case.

[Rick.Vanover]

Hi Adam, Double-jump is an extra step - I'm not wild about that. Getting images out of deduplication appliances is indeed a matter of rehydration and latency, even with mechanisms like DDBoost it still is the case.

[Dima P.]

Hello Adam,

Can you please clarify the type of tape job you are referring to? If you backing up files via file to tape job then how the encryption is set? If you are talking about vm backup files that are going to be written to tape media via backup to tape job please clarify how the source disk jobs are configured (do you have periodic full backup option enabled)? Thank you in advance.

[ninjaburn81]

Thanks guys for the responses...sorry the notifications must have gotten buried in my email.

If you backing up files via file to tape job then how the encryption is set? If you are talking about vm backup files that are going to be written to tape media via backup to tape job please clarify if how the source disk jobs are configured (do you have periodic full backup option enabled)?

This is a Backup-to-Tape job using GFS tape (weekly/monthly). I went at looked at the problem source job, which is all unstructured data (always a headache I know). I checked and the Storage is set to Incremental, with an Active Full created on the last Sunday of each month.

I'm wondering if I am handling this wrong? I could do a synthetic full every Saturday on the source job, then run the Backup-to-Tape on Sunday. OR just run a backup copy job every 7 days...but I'm not thrilled with that idea due to the amount of data.

FYI: tested without encryption this past weekend, basically no difference in time. So not the cause. Also, this is all on the same LAN, so speed is not a factor here.

[ers_kentwick]

Our setup is somewhat similar but we are getting faster throughput that your site.
Our setup is "Weekly backup (select VM)" to DD2500, Tape copy job is set to copy as backup files appear (schedule is Friday 17:00 to Monday 06:00), Tape library is IBM TS3310 with library managed encryption (Veeam encryption is not used), Tape drives are LTO-5.

Our job that started at 22:00 on 11/24 ran till 20:45 on 11/25. Read 3.1 TB from DD2500 and wrote 1.8 TB to tape.
So 3.1 TB read in 23 hours.

[ninjaburn81]

ers_kentwick, I'm assuming your source job is created a synthetic (or active) full?

[ninjaburn81]

So I'm definitely thinking it is the lack of a full backup prior to running the weekly tape backup. I checked the details of the last few jobs, and all the details of the tape job for the file shares note 'synthetic full' in the details, while the other source jobs are blank. I checked and all those source jobs have either active or synthetic fulls run at least weekly. I have changed up the backups to create a synthetic full on Saturday, and then run the weekly tape job on Sunday. I'll know this weekend if it helps!

[DominikM]

Any result on that?

I'm struggling with the same problem. Our main Backup Storage is a DD6800. Even though this device should perform way better than a DD2500, we explore relatively slow backup to tape speeds.
Did the change of the schedule of the synthetic full help?

[Dima P.]

Hello folks,

We are investigating the performance issue with synthesized full backup in GFS media pools and your logs will help us to get a clear picture of this problem. If you face this issue, kindly, open a support case and share the case ID in this thread. Thank you in advance.

[DominikM]

Here is my Case # 03334827