-
- Novice
- Posts: 5
- Liked: never
- Joined: Oct 20, 2015 12:55 pm
- Full Name: Jeremy Birch
- Contact:
Need help designing backup jobs (Tape and Data Domain)
I am looking for recommendations for my backup job settings as I am a little overwhelmed with the options I have. I am having issues where my Production_RAM job runs from Wednesday night until Friday morning when it's doing it's active full so I miss Thursday evening backups all together. I then also have the issue of my Friday evening backup (Production_TAPE) running 25 hours and then the (Production_RAM) doesn't run on Fridays so I don't have backups to the DD6200 for that day nor do I have it replicated to the other site. Plus I only have one week of retention due to it being on local disk.
FYI I had issues with slow copy jobs to tape when trying to copy from the Data Domain which is why I have a separate job.
Overview
TS3100 Tape Library with one LTO6 tape drive for VEEAM
EMC DD6200 Data Domain in Primary Site 15.7TB Usable Space (Replication is occurring between the data domains)
EMC DD6200 DR Site 15.7TB Usable Space
HP Proliant DL380 Gen8 with 29TB of local disk
What I have been doing:
Job Title: Production_RAM
Backup Repository: EMC DD6200 Data Domain in Primary Site
Restore Points to Keep: 60
Backup Mode: Incremental
Active Full Backup: Weekly on Wednesday
Schedule: Every Day at 9:00 PM
Runtime: Incremental-12 Hours, Full-36 Hours
Job Title: Production_TAPE
Backup Repository: HP Proliant DL380 Gen8 Local Disk
Restore Points to Keep: 1
Backup Mode: Incremental (Really never an incremental as I run it only on the same day I have the active full backup running)
Active Full Backup: Weekly on Friday
Schedule: Weekly on Friday at 6:00 PM
Secondary Target: Backup to Tape (TS3100 Tape Library)
Runtime: 25.5 Hours to the local disk, 15 Hours to Tape
So to sum it all up I would say I need my jobs designed so that I get a backup every evening no matter what day of the week and possibly decrease run times.
FYI I had issues with slow copy jobs to tape when trying to copy from the Data Domain which is why I have a separate job.
Overview
TS3100 Tape Library with one LTO6 tape drive for VEEAM
EMC DD6200 Data Domain in Primary Site 15.7TB Usable Space (Replication is occurring between the data domains)
EMC DD6200 DR Site 15.7TB Usable Space
HP Proliant DL380 Gen8 with 29TB of local disk
What I have been doing:
Job Title: Production_RAM
Backup Repository: EMC DD6200 Data Domain in Primary Site
Restore Points to Keep: 60
Backup Mode: Incremental
Active Full Backup: Weekly on Wednesday
Schedule: Every Day at 9:00 PM
Runtime: Incremental-12 Hours, Full-36 Hours
Job Title: Production_TAPE
Backup Repository: HP Proliant DL380 Gen8 Local Disk
Restore Points to Keep: 1
Backup Mode: Incremental (Really never an incremental as I run it only on the same day I have the active full backup running)
Active Full Backup: Weekly on Friday
Schedule: Weekly on Friday at 6:00 PM
Secondary Target: Backup to Tape (TS3100 Tape Library)
Runtime: 25.5 Hours to the local disk, 15 Hours to Tape
So to sum it all up I would say I need my jobs designed so that I get a backup every evening no matter what day of the week and possibly decrease run times.
-
- Novice
- Posts: 5
- Liked: never
- Joined: Oct 20, 2015 12:55 pm
- Full Name: Jeremy Birch
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
I should also mention I have 3 VEEAM Backup Proxies that do that Hot-Add for the backups.
-
- Product Manager
- Posts: 20389
- Liked: 2298 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
So, the first issue to address would be not job redesign, but rather full backup taking a lot of time. Can you tell me how many VMs are present inside backup job? What component is identified as a major bottleneck during full backup cycle? Is hot add mode indeed used during backup jobs (look for special [hot-add] metric in the corresponding job session statistics). Thanks.
-
- Novice
- Posts: 5
- Liked: never
- Joined: Oct 20, 2015 12:55 pm
- Full Name: Jeremy Birch
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
Hi v.Eremin, you might be right about just reducing the time it takes for the full backup to complete. I currently have 104 virtual machines backing up using hot-add through the veeam virtual proxies to the DD6200 using DDBOOST. I've removed the server names for security sake.
Veeam Backup & Replication 8.0.0.2084
Code: Select all
Backup job: Production_RAM (Full)
at 5/1/2014 7:06:46 AM.Error
104 of 104 VMs processed
Wednesday, September 9, 2015 9:00:05 PM
Success97Start time9:00:05 PMTotal size15.8 TBBackup size11.0 TB
Warning2End time5:10:44 AM (+2)Data read11.3 TBDedupe1.4x
Error5Duration32:10:38 Transferred10.4 TBCompression1.0x
Details
NameStatusStart timeEnd timeSizeReadTransferredDurationDetails
Success9:06:31 PM9:29:38 PM60.0 GB59.1 GB59.1 GB0:23:07
Success9:06:31 PM10:26:11 PM300.0 GB165.6 GB150.9 GB1:19:40
Success9:12:07 PM3:14:46 AM (+1)1.2 TB1018.0 GB1014.0 GB6:02:38
Success9:27:52 PM10:26:36 PM50.0 GB40.8 GB40.8 GB0:58:44
Success9:28:02 PM10:36:43 PM60.0 GB44.0 GB41.3 GB1:08:41
Success10:01:20 PM2:28:25 AM (+1)810.0 GB756.1 GB738.6 GB4:27:04
Success10:24:01 PM2:17:58 AM (+1)60.0 GB25.6 GB24.0 GB3:53:56
Error10:24:07 PM1:15:31 AM (+1)0.0 KB0.0 KB0.0 KB2:51:24
Success12:05:34 AM2:47:19 AM75.0 GB25.4 GB22.8 GB2:41:45
Success12:14:42 AM3:35:33 AM130.0 GB128.2 GB111.2 GB3:20:51
Success2:08:20 AM8:14:20 AM860.0 GB488.1 GB463.9 GB6:05:59
Success2:17:53 AM3:59:04 AM150.0 GB142.2 GB102.8 GB1:41:10
Success2:28:16 AM4:04:43 AM67.5 GB39.7 GB36.0 GB1:36:26
Success2:47:31 AM4:24:06 AM60.0 GB59.0 GB59.0 GB1:36:35
Success3:24:22 AM4:38:30 AM60.0 GB45.8 GB45.2 GB1:14:08
Success3:26:58 AM5:43:24 AM120.0 GB51.0 GB43.8 GB2:16:25
Success3:38:11 AM5:32:02 AM50.0 GB48.0 GB27.3 GB1:53:51
Success3:59:07 AM3:54:40 PM1.5 TB1.2 TB1.1 TB11:55:32
Success4:37:22 AM9:03:12 AM60.0 GB20.2 GB18.7 GB4:25:50
Success4:37:22 AM9:02:25 AM60.0 GB33.2 GB31.0 GB4:25:02
Success5:18:39 AM9:02:53 AM250.0 GB38.8 GB38.7 GB3:44:14
Success7:38:20 AM9:35:29 AM74.5 GB70.6 GB66.8 GB1:57:09
Success8:02:27 AM9:52:05 AM100.0 GB96.0 GB67.1 GB1:49:37
Error8:13:57 AM9:36:58 AM67.4 GB67.4 GB64.1 GB1:23:01
Success8:36:59 AM10:01:41 AM50.0 GB37.5 GB37.5 GB1:24:42
Success9:03:02 AM10:02:10 AM60.0 GB26.4 GB25.0 GB0:59:08
Success9:10:54 AM10:20:04 AM120.0 GB51.8 GB44.3 GB1:09:09
Success9:35:18 AM10:41:39 AM90.0 GB71.3 GB69.7 GB1:06:21
Success9:51:52 AM11:04:09 AM120.0 GB88.2 GB60.3 GB1:12:16
Success9:53:08 AM10:59:07 AM120.0 GB83.5 GB58.0 GB1:05:58
Success9:53:29 AM11:04:28 AM100.0 GB92.0 GB92.0 GB1:10:59
Success10:01:16 AM11:18:06 AM50.0 GB35.6 GB33.6 GB1:16:50
Success10:13:04 AM12:36:12 PM136.7 GB134.7 GB134.3 GB2:23:07
Success10:19:47 AM11:32:15 AM100.0 GB37.0 GB33.3 GB1:12:27
Success10:42:58 AM11:35:11 AM120.0 GB28.1 GB25.8 GB0:52:12
Success11:04:19 AM1:22:13 AM (+1)2.1 TB1.8 TB1.7 TB14:17:53
Success11:17:02 AM11:36:35 AM18.0 GB8.1 GB8.1 GB0:19:32
Error11:17:17 AM12:34:25 PM224.6 GB221.4 GB196.7 GB1:17:08
Success11:32:01 AM12:33:26 PM9.6 GB9.5 GB1.3 GB1:01:24
Success11:35:03 AM1:40:35 PM70.0 GB51.7 GB46.2 GB2:05:32
Success11:45:31 AM1:40:39 PM120.0 GB55.8 GB55.7 GB1:55:07
Success12:19:31 PM1:40:36 PM60.0 GB24.3 GB21.4 GB1:21:05
Success12:33:15 PM2:01:32 PM60.0 GB33.2 GB21.8 GB1:28:17
Success12:33:15 PM1:40:08 PM60.0 GB30.2 GB28.7 GB1:06:53
Success1:04:36 PM2:29:20 PM100.0 GB96.0 GB68.1 GB1:24:43
Success1:04:36 PM2:01:57 PM50.0 GB31.6 GB29.8 GB0:57:20
Success1:27:20 PM2:34:36 PM30.0 GB24.4 GB24.4 GB1:07:15
Success1:27:31 PM2:49:39 PM64.0 GB61.3 GB58.0 GB1:22:08
Success1:40:00 PM2:34:15 PM160.0 GB22.0 GB20.4 GB0:54:14
Success2:21:54 PM3:03:44 PM20.0 GB16.0 GB2.4 GB0:41:50
Success2:28:56 PM3:23:02 PM80.0 GB42.5 GB40.8 GB0:54:05
Success2:34:08 PM4:56:52 PM380.0 GB282.4 GB271.6 GB2:22:43
Error2:49:23 PM3:44:04 PM160.0 GB61.8 GB59.4 GB0:54:40
Success3:23:33 PM4:57:22 PM60.0 GB21.7 GB19.1 GB1:33:49
Success3:42:22 PM5:03:46 PM100.0 GB96.0 GB62.4 GB1:21:23
Success3:42:48 PM5:47:46 PM110.0 GB75.0 GB73.7 GB2:04:58
Success3:42:48 PM5:02:13 PM100.0 GB20.9 GB19.7 GB1:19:25
Success4:18:07 PM5:35:47 PM50.0 GB19.0 GB19.0 GB1:17:40
Success4:56:37 PM5:36:08 PM110.0 GB42.7 GB39.9 GB0:39:30
Success4:56:47 PM5:43:50 PM30.0 GB22.7 GB22.7 GB0:47:03
Success5:31:56 PM7:14:04 PM260.0 GB96.1 GB88.1 GB1:42:08
Success5:32:16 PM7:04:06 PM200.0 GB86.5 GB86.5 GB1:31:50
Success5:33:17 PM7:14:47 PM60.0 GB31.8 GB29.1 GB1:41:30
Success5:33:42 PM7:14:27 PM30.0 GB23.7 GB23.1 GB1:40:44
Success5:44:06 PM8:53:56 PM30.0 GB21.4 GB21.4 GB3:09:50
Success6:51:37 PM8:58:35 PM30.0 GB28.8 GB28.0 GB2:06:57
Success6:51:38 PM8:57:37 PM70.0 GB62.4 GB62.4 GB2:05:59
Success6:51:58 PM8:58:02 PM60.0 GB28.8 GB26.9 GB2:06:03
Success7:02:10 PM9:14:43 PM150.0 GB134.6 GB123.5 GB2:12:32
Warning7:14:19 PM9:56:20 PM60.0 GB20.6 GB20.6 GB2:42:00
Success7:41:47 PM10:38:25 PM50.0 GB46.0 GB46.0 GB2:56:38
Success8:53:22 PM10:38:45 PM120.0 GB117.0 GB71.0 GB1:45:23
Success8:53:42 PM10:48:59 PM120.0 GB58.1 GB55.8 GB1:55:16
Success10:14:19 PM11:43:55 PM30.0 GB24.4 GB24.4 GB1:29:35
Success10:35:36 PM11:59:22 PM100.0 GB66.8 GB60.4 GB1:23:46
Error10:35:51 PM11:59:41 PM96.0 GB89.4 GB86.3 GB1:23:49
Success10:36:07 PM12:19:01 AM (+1)100.0 GB49.4 GB49.4 GB1:42:54
Success10:49:01 PM1:10:35 AM (+1)300.0 GB68.9 GB68.9 GB2:21:34
Success11:43:49 PM1:12:07 AM (+1)50.0 GB26.9 GB25.3 GB1:28:18
Success11:49:41 PM1:12:07 AM (+1)60.0 GB21.1 GB19.7 GB1:22:25
Success11:50:12 PM1:11:44 AM (+1)60.0 GB25.4 GB25.4 GB1:21:32
Success11:58:55 PM1:46:14 AM (+1)120.0 GB105.1 GB95.1 GB1:47:19
Success12:25:07 AM1:46:38 AM80.0 GB30.1 GB30.0 GB1:21:31
Success12:41:08 AM1:57:31 AM80.0 GB52.0 GB51.8 GB1:16:23
Success12:41:08 AM1:57:09 AM60.0 GB30.5 GB27.8 GB1:16:01
JBMANW7Success12:41:23 AM1:57:32 AM60.0 GB58.0 GB54.8 GB1:16:08
Warning1:10:20 AM2:01:05 AM60.0 GB20.9 GB19.5 GB0:50:44
Success1:22:05 AM2:00:49 AM30.0 GB24.4 GB24.4 GB0:38:44
Success1:22:10 AM2:01:05 AM60.0 GB34.3 GB23.0 GB0:38:54
Success1:42:15 AM3:05:12 AM70.0 GB69.5 GB68.4 GB1:22:57
Success1:42:35 AM3:47:59 AM280.0 GB252.7 GB239.1 GB2:05:24
Success1:57:06 AM3:05:14 AM80.0 GB78.2 GB39.8 GB1:08:07
Success1:57:12 AM3:20:42 AM100.0 GB96.0 GB59.2 GB1:23:30
Success2:33:36 AM3:47:33 AM75.0 GB35.6 GB35.6 GB1:13:56
Success2:33:41 AM3:48:31 AM60.0 GB37.7 GB34.0 GB1:14:49
Success2:33:57 AM4:04:31 AM160.0 GB110.3 GB109.1 GB1:30:34
Success3:03:26 AM4:35:16 AM80.0 GB75.2 GB63.7 GB1:31:50
Success3:09:33 AM5:10:08 AM545.0 GB472.9 GB400.2 GB2:00:35
Success3:47:54 AM5:05:31 AM70.0 GB48.1 GB46.0 GB1:17:37
Success3:48:04 AM4:59:00 AM100.0 GB84.9 GB84.8 GB1:10:56
Success4:01:33 AM5:07:21 AM60.0 GB21.7 GB20.3 GB1:05:48
Success4:01:43 AM5:09:52 AM120.0 GB98.2 GB84.2 GB1:08:08
Success2:01:38 PM3:03:17 PM30.0 GB25.0 GB25.0 GB1:01:38
Success9:29:32 PM10:26:09 PM0.0 KB0.0 KB0.0 KB0:56:36
Veeam Backup & Replication 8.0.0.2084
-
- Novice
- Posts: 5
- Liked: never
- Joined: Oct 20, 2015 12:55 pm
- Full Name: Jeremy Birch
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
Also, the target is identified as the bottleneck. Here's a copy of what I normally see for full backup job (not the one that goes to local disk and off to tape but the one that goes to the Data Domain.)
7/31/2015 4:45:10 AM :: Load: Source 24% > Proxy 22% > Network 66% > Target 73%
7/31/2015 4:45:10 AM :: Load: Source 24% > Proxy 22% > Network 66% > Target 73%
-
- Influencer
- Posts: 12
- Liked: 14 times
- Joined: Jan 06, 2015 10:46 pm
- Full Name: Ethan Cain
- Location: Miramr, FL
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
Going to DD, target will always be the bottleneck unless it's on a 10/100 network.
It looks like the overall average on the Active full is ~100MB/s not including the time for each VM to do pre and post processing, etc.
You have a DD620? Correct? Was there ever a 6200?
Personally, I would have that 104 VMs broken into 4 jobs to spread it out a bit. DataDomain and Single Stream writing sucks, and as far as I have seen, on a normal Active Full / Inc DDBoost isn't mind blowing in performance in the current state.
I have jobs with ~100 VMs going to a very large DD, that don't exceed 16-30MB/s but I can run 20 of those jobs at around those speeds.
It looks like the overall average on the Active full is ~100MB/s not including the time for each VM to do pre and post processing, etc.
You have a DD620? Correct? Was there ever a 6200?
Personally, I would have that 104 VMs broken into 4 jobs to spread it out a bit. DataDomain and Single Stream writing sucks, and as far as I have seen, on a normal Active Full / Inc DDBoost isn't mind blowing in performance in the current state.
I have jobs with ~100 VMs going to a very large DD, that don't exceed 16-30MB/s but I can run 20 of those jobs at around those speeds.
Retired 'Cloud Admiral'. Might actually be on a ship.
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Need help designing backup jobs (Tape and Data Domain)
Ethan is spot on. Besides the fact that we do not recommend using any deduplicating storage as a primary backup repository (it is a perfect secondary backup target for long-term backups archival), with a single job you're not fully utilizing the available I/O it provides. Multiple jobs will create multiple I/O streams resulting in better ingestion rate.
Who is online
Users browsing this forum: DVashkis, Gostev, GregorS, ssimakov and 169 guests