Best Practice for MS Server 2012 DeDup Repo

Availability for the Always-On Enterprise

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Vitaliy S. » Fri May 24, 2013 8:30 pm

Oh, ok! I would appreciate if you could update this topic with your findings (setting throughput duration to 0).
Vitaliy S.
Veeam Software
 
Posts: 19104
Liked: 1055 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Sun May 26, 2013 7:49 pm

Just tested, throughput duration can only be set between 1-24 hours.
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Vitaliy S. » Mon May 27, 2013 6:40 am

Makes sense...
Vitaliy S.
Veeam Software
 
Posts: 19104
Liked: 1055 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby mongie » Mon May 27, 2013 8:52 pm 2 people like this post

Alexander - What I've done is... go into task manager. Export all the built in dedupe jobs (they're under microsoft / deduplication) and then disable them. Then I import the jobs I just exported, and you can customise them. The reason for the import/export is that the settings you change seem to be re-set after a while if you just edit the built in jobs.

I've turned off the "stop after x hours" setting, and that allows jobs to run for days / weeks if required.

Its really just running an executable file with some switches. You can also edit the switches to enable /full, /priority high, /backoff (which stops processing when it detects IO) and edit the memory usage target (I set mine to 75.

I hope this helps.
mongie
Expert
 
Posts: 140
Liked: 20 times
Joined: Mon May 16, 2011 4:00 am
Location: Brisbane, Australia
Full Name: Alex Macaronis

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Mon May 27, 2013 10:14 pm

Thanks mongie for the tips, I'm going to try changing the schedule options through powershell instead commands and see if they still get re-set.

If not I will be doing your task scheduler hack. What does the /full switch do? Can't find anything on that.
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby mongie » Tue May 28, 2013 2:40 am

Imagine /full being a full backup and not /full being an incremental.


I'm pretty sure thats the difference.
mongie
Expert
 
Posts: 140
Liked: 20 times
Joined: Mon May 16, 2011 4:00 am
Location: Brisbane, Australia
Full Name: Alex Macaronis

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby v.Eremin » Tue May 28, 2013 9:03 am

I'm going to try changing the schedule options through powershell instead commands and see if they still get re-set.

There is also an article how deduplication schedule can be changed via PS and that is a nice place to start your acquaintance with deduplication PS commandlets.

Thanks.
v.Eremin
Veeam Software
 
Posts: 12606
Liked: 910 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby mongie » Tue May 28, 2013 10:11 am

A tip that I (novice at power shell) found was to use the | fl command after my command lets.

E.g. get-dedupestatus | fl

and

Get-dedupejob | fl
mongie
Expert
 
Posts: 140
Liked: 20 times
Joined: Mon May 16, 2011 4:00 am
Location: Brisbane, Australia
Full Name: Alex Macaronis

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby v.Eremin » Tue May 28, 2013 10:32 am

A tip that I (novice at power shell) found was to use the | fl command after my command lets.

Yep, fl or Format-List is a nice commandlet to control the output of your main script.
v.Eremin
Veeam Software
 
Posts: 12606
Liked: 910 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Thu May 30, 2013 9:18 pm

Posted on MS forum about my dedup concerns on files larger than 2TB and the 24 hour limit on throughput scheduling and got this reply from development guys on the Server 2012 dedup feature: http://social.technet.microsoft.com/For ... 6d9604f06/

The number applies to how much can be deduplicated on a single volume.
If the machine has multiple cores and sufficient memory then Dedup will schedule jobs to run in parallel, one per volume, so the overall throughout of the machine could be 4x, 8x or even 16x than the number you quoted.
If you have a choice when you provision a machine intended for Dedup is to provision with multiple large volumes rather than one huge volume (e.g. create 16 4TB volumes each rather than a single 64TB volume), Dedup does not have at the moment the ability to run multiple jobs in parallel on the same volume.


And I don't know what to make of that reply really. Should I split my 16 TB volume to 4 volumes instead? How should I then configure my veeam job to land on the different volumes? My full backup is 3,5 TB.

So confusing...
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby tsightler » Thu May 30, 2013 9:40 pm

Can you clarify what exactly you are attempting to accomplish? Is your backup 3.5TB with Veeam compression disabled? Are you using reverse incremental? How many restore points are you trying to keep on disk?

It's well known that the limitation of Windows 2012 dedupe processing is per-volume, so yes you could easy split your 16TB volume into 2x 8TB volumes and split you backups into two jobs that were roughly equal in size (1.75TB each), and you'd be good, but without knowing more about what exactly your goal is it's hard to decide if that's the best option as it might be that you'd be better off simply using reverse incremental backup and not using Windows 2012 dedupe at all.
tsightler
Veeam Software
 
Posts: 4663
Liked: 1683 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Thu May 30, 2013 9:53 pm

I have veeam compression enabled and dedupe friendly option. My config looks like this:

1 veeam backup job, 56 vm, Forward incremental, active full every saturday, 30 restore points.
Full backup size has actually grown to 3,9 TB and incremental between 500-700GB.
16,4 TB Volume with Raid 5, 7 x 3TB 7,2K DAS on Physical server with Server 2012 Dedupe configured.
Right now I have only 27% dedup rate and 4,29 TB saved.

Any suggestion to what I should do differently?
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby tsightler » Thu May 30, 2013 10:35 pm

If I had that configuration and was only interested in keeping 30 days of backups, I'd honestly probably just use reverse incremental and not use Windows 2012 dedupe at all. Veeam with normal compression will likely redeuce your backup to <2TB, and your incrementals to <300GB on average. That means that 30 days of backups would only need ~11TB anyway, and you said you have a 16TB volume, and that's assuming only a 2:1 compression ratio, most environments see closer to 3:1 which would out it closer to 9TB with Veeam alone.

On the other hand, if you just want to use Windows 2012 dedupe and forward incremental, it sounds likes it's already working reasonably well (it's saved 4.29TB). Do you have all 5 full backups yet? That's where most of the dedupe will come from. Dedupe friendly compression will reduce your dedupe ratios by quite a bit and normally I would suggest using no compression at all if you are interested in getting the absolute best dedupe ratios.
tsightler
Veeam Software
 
Posts: 4663
Liked: 1683 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Fri May 31, 2013 7:12 am

So change to Reverse incremental, turn on better Veeam compression and turn off Server 2012 dedupe would yield better storage savings for me?

I read that the reverse incremental will require 3 x IOPS requirement compared to forward incremental. I'm also worried about the backup window. Right now it is taking around 10-11 hours for a full. How long would a reverse incremental take? Also if I turn off Server 2012 dedupe, then my other files (backup for physical servers with backup exec) on the volume would not be able to dedupe.
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby baatch » Tue Jun 04, 2013 3:25 pm

On the compression level, is there any reason not to do Optimal or Extreme instead of dedupe friendly for Server 2012 dedup?
baatch
Enthusiast
 
Posts: 30
Liked: 4 times
Joined: Thu May 16, 2013 12:52 am
Full Name: Alexander

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Google Feedfetcher, Yahoo [Bot] and 21 guests