Comprehensive data protection for all workloads
Post Reply
eringuet
Novice
Posts: 3
Liked: never
Joined: Dec 18, 2013 6:21 pm
Full Name: Etienne Ringuet
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by eringuet » Jan 20, 2014 2:05 pm

Very interesting post Peter, thanks for sharing.
5. Based off of the post below from Yizhar, I found the best way to handle dedupe was to carve up 2 big volumes. Volume 1 for a Hot Landing zone for recent backups (7 Days or so) and Volume 2 for long term archiving. The long term archive zone can be deduped via schedule to run 24/7 and use 50% of server ram. It should be able to dedup 8.4 TB per week using a rate of 50GB per hour. Dedup is much slower on .vbk's over 1 TB. By using this method, you don't have to break your backup jobs into smaller jobs and you will get global deduplication across VBK files. We have our backups jobs setup by Server OS.
Can anyone expend on this strategy? I've been running Veeam for a week and I'm not sure how to go about configuring this.

Thanks

foggy
Veeam Software
Posts: 18131
Liked: 1536 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy » Jan 20, 2014 2:34 pm

Basically, you need to create two different backup repositories. The first one will be a target for the regular backup job and will contain recent backups for fast operational restores. Then the backup copy job will copy backups from the first repository to the second one, with deduplication enabled, for longer-term archival. If you need instructions on configuring backup repositories and jobs, I recommend to review the corresponding user guide sections first.

NightBird
Service Provider
Posts: 175
Liked: 32 times
Joined: Apr 28, 2009 8:33 am
Location: Strasbourg, FRANCE
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by NightBird » Jan 20, 2014 4:27 pm 1 person likes this post

What about the synthetic full generated by backup copy job GFS ==> Killing the windows 2012 dedup process ?
Once a week a full .vbk will be syntheticaly generated, ouch... with an ingestion rate of 50GB/h

paps40
Enthusiast
Posts: 26
Liked: 13 times
Joined: Dec 12, 2011 4:10 pm
Full Name: Peter Pappas
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by paps40 » Jan 20, 2014 5:33 pm

Great Point! I misunderstood a backup copy job. I assumed it would simply copies files from A to B but that is not the case. Because of that, I have resorted back to robocopy to copy backups that are 1 day old from landing zone to archive zone. It's more important for us to utilize server 2012 dedup on the archive zone while sending the most recent un-deduplicated active fulls from the landing zone to tape each week.

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » Jan 21, 2014 7:17 am

Yes, you're right. The backup copy job doesn't t copy backup file as a whole, but rather synthetically creates required restore points in "target" location from VM data in source backup repositories.

As to GFS restore points, they will be created from the existing backup copy chain. Once the retention period of backup copy job is exceeded, the full backup starts to be moved forward on daily basis ("transformation"). When it reaches the specified day (GFS schedule), the backup copy job will create an independent full restore point, using the existing full backup for it.

Thanks.

namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 » May 12, 2014 4:21 pm

We have a requirement to save a monthly restore point, offsite, for 7 years. I was debating using the copy job (writing to the DR site) with monthly archive points, and then deduping them to save disk space, then it occurred to me, why bother with this, why can't i just use a monthly backup job that is the same selection list as my main job, and setting 84 (7 years) restore points?

The only advantage i can think of using the copy job is that i'll have multiple full backups, in case something ever happened to the main VBK.

Are there any other advantages i'm not considering, and if i was to do the monthly job, should i use forward incremental, with extreme compression ( i know dedupe may be pointless with this setting) or no compression and dedupe. Comments please :)

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » May 12, 2014 4:45 pm

The only advantage i can think of using the copy job is that i'll have multiple full backups, in case something ever happened to the main VBK.
By default, backup copy job preserves one full backup and chain of dependent increments. If you want to have multiple full backups, you will have to specify GFS retention scheme.
If i was to do the monthly job, should i use forward incremental, with extreme compression ( i know dedupe may be pointless with this setting) or no compression and dedupe.
In order to guarantee max deduplcation rates, you should disable native VB&R compression/deduplication.

With monthly backup job you can probably stick to "always full backup" scenario, as the size of changes generated throughout the month might be approximately equal to the size of new full backup. Another advantage of "always full backup" scenario is that you will have multiple independent full backups.

Thanks.

namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 » May 12, 2014 4:53 pm

to clarify, you mean i should uncheck "enable inline data deduplication" ?

namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 » May 12, 2014 5:01 pm

and are you recommending creating a separate "monthly job" and using the active full method, or using the backup copy job with an every 30 day run time?
Thanks

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » May 13, 2014 8:56 am

You can leave inline deduplication as is, but disable compression and set LAN target as the storage optimization level.

It's up to chose either additional backup job or backup copy job. Both scenarios are popular among our customers. I was trying to say that if you finally stick to backup job, then, it will make sense to use "always full" approach, as this will give independent full backups, meanwhile, the additional amount of space needed will be negligible, comparing to monthly worth increments.

Thanks.

jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw » May 28, 2014 11:48 am

Our Veeam backups were previously set to reversed incrementals, with no de-dup options selected - such as LAN target and compression disabled.

We have now enabled Dedup, however can anyone recommend what to do with existing backups? Are we better off archiving the old backups, and starting fresh with the recommended dedup settings to achieve maximum dedup results?

foggy
Veeam Software
Posts: 18131
Liked: 1536 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy » May 28, 2014 1:02 pm

Do you mean you have enabled Data Deduplication feature on the repository or just changed Veeam B&R job settings to the ones recommended when storing backups on a deduplicated volume?

Anyway, old backups will eventually fall out of the retention period and be deleted, so not a big deal. Do you have long chain of increments?

jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw » May 28, 2014 2:16 pm

foggy wrote:Do you mean you have enabled Data Deduplication feature on the repository or just changed Veeam B&R job settings to the ones recommended when storing backups on a deduplicated volume?

Anyway, old backups will eventually fall out of the retention period and be deleted, so not a big deal. Do you have long chain of increments?
We have enabled deduplication on a newly rebuilt server. Veeam was previously running on Server 2008 R2 - so we just rebuilt the server with 2012 R2, and re-attached our RDM disk. Installed the deduplication feature, and enabled. So far, we have only seen a 9% deduplication rate. These increments were previously written to the non-deduplicated disk with non-deduplication friendly settings.

I suppose over time, we should see an increased deduplication rate? I'd like to avoid re-starting backups.

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » May 28, 2014 2:21 pm

I suppose over time, we should see an increased deduplication rate?
The best deduplication rates will be achieved, if you switch to forward incremental backup mode. Thanks.

jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw » May 28, 2014 3:37 pm

I have made this change in the backup job, and set it to Incremental. I have also set an Active Full to take place once a week. Looking forward to results!

VladV
Expert
Posts: 223
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

[MERGED] : Synology NAS iSCSI attached Enable 2012 Dedup

Post by VladV » Jul 06, 2014 9:20 pm

I am tinkering with the idea of enabling deduplication on the primary backup target which is a Synology DS1512+ attached to the Veeam server (2012 R2 VM) as an iSCSI target. Given that Microsoft states that their dedup has a minimal impact on performance wouldn't this be a good idea or am I missing something.

Regards,
Vlad

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » Jul 07, 2014 8:05 am

Hi, Vlad, your post has been merged into existing discussion about best practice of using MS deduplication on primary backup repository. Please, see answers provided above.

In general, it's still recommended to utilize dedupe appliances as secondary repository, not the primary one. If you're still going to use it as a primary repository, then, set appropriate dedupe settings ("Dedupe data older than") in order to guarantee fast access to recent backup data.

Thanks.

ChrisJ83
Influencer
Posts: 17
Liked: 1 time
Joined: Aug 11, 2014 7:44 am
Full Name: Chris Johnson
Contact:

[MERGED] : Server 2012 R2 Deduplication and Veeam deduplicat

Post by ChrisJ83 » Aug 20, 2014 7:48 am

Hi

I have contacted Veeam support about this but I don't feel the support representative understood my query correctly so I thought I would see if anyone on the forum had any advice.

case 00620488

I am in the process of setting up a new backup server to run Veeam and it will run Server 2012 R2, I will then copy and archive backups onto a Synology SAN but I don't think that bit matters for this post.

I was thinking of enabling Server 2012 R2 deduplication on the backup volume and then keep inline data deduplication ticked however support seemed to insinuate I could only enable one or the other.

I personally thought Veeam would do its thing and then store the backup with its own inline deduplication and Windows would then provide another level of deduplication which works on small chunks of data. 4 full backups would pretty much become 1 full backup depending on the amount of changed data.

Any help/advie/experience with this appreciated.

Cheers
Chris

Gostev
SVP, Product Management
Posts: 24621
Liked: 3465 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Server 2012 R2 Deduplication and Veeam deduplication

Post by Gostev » Aug 20, 2014 8:03 am

Hi! You can keep both enabled, in fact I would recommend that in terms of incremental backup performance. Having Veeam dedupe enabled will certainly not have any impact on Windows dedupe. Thanks!

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » Aug 20, 2014 8:11 am

Hi Chris,

Your post has been merged into existing discussion regarding similar matter, so please take a look at the answers provided above. In general, you can leave inline deduplication enabled, but disable compression for achieving better deduplication rates.

Thanks.

nunciate
Expert
Posts: 179
Liked: 23 times
Joined: May 21, 2013 9:08 pm
Full Name: Alan Wells
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by nunciate » Aug 25, 2014 5:46 pm

Here is my setup if you are interested in the numbers.
HP Server with SAS attached storage. 3 arrays with 12x 2Tb drives in RAID5 with 1 hotspare each.
In Windows they come up as 3 drives at about 18Tb each.
1 Drive has Deduplicaiton turned off. I have several very large backups files on that on of them is 9Tb big so I don't bother deduping those.
The other 2 drives have dedupe truned on and I get very good deduplication rates.
Drive 1 shows actual size of all files stored on the array at 37.6Tb. The disk shows only 15.9Tb used however.
Drive 2 shows actual size of all files stored on the array at 26.4Tb. The disk shows only 9.06Tb used.
These numbers don't reflect the deduplication that is still running from my weekend fulls so the space on disk will actully go up over the next day or 2 freeing up space for the next weekend fulls.

I can tell you that I started out using mostly Synthetic Full with reverse incremental. My deduplication settings are set process files older than 1 day. So the most recent files always get deduped.
At first reverse incrementals seemed to work fine. However, over time I noticed the weekly trnasforms started taking longer and longer. One backup with a vbk of less than 800Gb took 62 hours to process.
I since switched all of those backups to Incremental with a weekly active full. The numbers above reflect that setting and the backups run much faster now. The trick is to run the full backups so that you have time to deduplicate before filling up the drives. I run my fulls across several nights and all seems well.

In all I am backing up almost 50Tb of data across Thursday, Friday & Saturday nights. I am able to backup and tape out everything before Sunday evening with no issues.

gggggggg
Influencer
Posts: 15
Liked: never
Joined: Aug 26, 2014 1:50 am
Full Name: greg
Contact:

[MERGED] Windows v Veeam Deduplication

Post by gggggggg » Aug 26, 2014 2:02 am

Hello, I am just planning our new Veeam install on Win 2012 R2.

To confirm I have this right:
[*]Veeam will de-duplicate at backup time, resulting in smaller VBK files
[*]Veeam will only de-duplicate servers within the same backup job
[*]Windows de-duplication will backup files on the disk, after Veeam has finished with the VBK

Sound right??

New question. In the job Advanced settings, Storage, Compression. DeDupe Friendly v others, will that have any impact on the deduplication ratios, or will it just use more RAM/CPU in the deduplication process?

thanks

foggy
Veeam Software
Posts: 18131
Liked: 1536 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy » Aug 26, 2014 10:51 am

Greg, your understanding is correct. I've merged your post into existing discussion regarding Windows 2012 deduplication, please review for more details. Thanks.

Shestakov
Veeam Software
Posts: 6910
Liked: 700 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by Shestakov » Aug 26, 2014 10:53 am

Hello Greg,
Greg wrote:Veeam will de-duplicate at backup time, resulting in smaller VBK files
True. You can choose storage optimization options depending on the type of storage you select as a backup target.
Greg wrote:Veeam will only de-duplicate servers within the same backup job
Correct. Also deduplication eliminates blocks of free space, which decreases the size of the created backup file.
Greg wrote:Windows de-duplication will backup files on the disk, after Veeam has finished with the VBK
I`m a bit confused here. VBK is a backup file itself which will be stored on your repository after backup job is finished. Deduplication is a mechanism to make backup process more efficient, decrease .vbk file size.
Greg wrote:In the job Advanced settings, Storage, Compression. DeDupe Friendly v others, will that have any impact on the deduplication ratios, or will it just use more RAM/CPU in the deduplication process?
Yes, deduplication ratios depend on compression level. Also type of data on disks tampers deduplication and compression level.
Thanks.

veremin
Product Manager
Posts: 16778
Liked: 1405 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin » Aug 26, 2014 10:55 am

nunciate wrote:At first reverse incrementals seemed to work fine. However, over time I noticed the weekly trnasforms started taking longer and longer. One backup with a vbk of less than 800Gb took 62 hours to process.
This is expected, as already deduped .vbk file has to be rehydrated in order for changes to be injected inside it. Thus, it's always recommended to put forward incremental mode with active full backup that doesn't touch any restore points and, thus, achieve better deduplication rates across multiple full backups. Thanks.

jadams159
Enthusiast
Posts: 80
Liked: 4 times
Joined: Apr 16, 2012 11:44 am
Full Name: Justin Adams
Location: United States
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jadams159 » Nov 25, 2014 7:47 pm

Sightler, did you ever complete that whitepaper on Server 2012 dedupe you mentioned earlier in the thread?

Gostev
SVP, Product Management
Posts: 24621
Liked: 3465 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by Gostev » Nov 26, 2014 11:58 pm

AFAIK not... I think he hates it now, after seeing people struggle with one so much :D

pepijn
Novice
Posts: 7
Liked: never
Joined: Jul 30, 2013 8:38 am
Full Name: Pepijn Deneut
Contact:

[MERGED] Windows server 2012 deduplication on backup reposit

Post by pepijn » Feb 12, 2015 8:05 am

Hello,

I want to implement windows server 2012 deduplication on our backup repository.

Is this officially supported by Veeam?

I have seen the webinar on http://hyperv.veeam.com/winserver-dedup ... joep-2069/.

Are these still the best practices?

thanks

foggy
Veeam Software
Posts: 18131
Liked: 1536 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy » Feb 12, 2015 9:33 am

Totally supported. Please look through this thread for some hints regarding enabling dedupe on the repository.

pepijn
Novice
Posts: 7
Liked: never
Joined: Jul 30, 2013 8:38 am
Full Name: Pepijn Deneut
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by pepijn » Feb 12, 2015 10:08 am

Thanks

Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 39 guests