Comprehensive data protection for all workloads
Post Reply
eringuet
Novice
Posts: 3
Liked: never
Joined: Dec 18, 2013 6:21 pm
Full Name: Etienne Ringuet
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by eringuet »

Very interesting post Peter, thanks for sharing.
5. Based off of the post below from Yizhar, I found the best way to handle dedupe was to carve up 2 big volumes. Volume 1 for a Hot Landing zone for recent backups (7 Days or so) and Volume 2 for long term archiving. The long term archive zone can be deduped via schedule to run 24/7 and use 50% of server ram. It should be able to dedup 8.4 TB per week using a rate of 50GB per hour. Dedup is much slower on .vbk's over 1 TB. By using this method, you don't have to break your backup jobs into smaller jobs and you will get global deduplication across VBK files. We have our backups jobs setup by Server OS.
Can anyone expend on this strategy? I've been running Veeam for a week and I'm not sure how to go about configuring this.

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy »

Basically, you need to create two different backup repositories. The first one will be a target for the regular backup job and will contain recent backups for fast operational restores. Then the backup copy job will copy backups from the first repository to the second one, with deduplication enabled, for longer-term archival. If you need instructions on configuring backup repositories and jobs, I recommend to review the corresponding user guide sections first.
NightBird
Expert
Posts: 242
Liked: 57 times
Joined: Apr 28, 2009 8:33 am
Location: Strasbourg, FRANCE
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by NightBird » 1 person likes this post

What about the synthetic full generated by backup copy job GFS ==> Killing the windows 2012 dedup process ?
Once a week a full .vbk will be syntheticaly generated, ouch... with an ingestion rate of 50GB/h
paps40
Enthusiast
Posts: 31
Liked: 13 times
Joined: Dec 12, 2011 4:10 pm
Full Name: Peter Pappas
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by paps40 »

Great Point! I misunderstood a backup copy job. I assumed it would simply copies files from A to B but that is not the case. Because of that, I have resorted back to robocopy to copy backups that are 1 day old from landing zone to archive zone. It's more important for us to utilize server 2012 dedup on the archive zone while sending the most recent un-deduplicated active fulls from the landing zone to tape each week.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

Yes, you're right. The backup copy job doesn't t copy backup file as a whole, but rather synthetically creates required restore points in "target" location from VM data in source backup repositories.

As to GFS restore points, they will be created from the existing backup copy chain. Once the retention period of backup copy job is exceeded, the full backup starts to be moved forward on daily basis ("transformation"). When it reaches the specified day (GFS schedule), the backup copy job will create an independent full restore point, using the existing full backup for it.

Thanks.
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 »

We have a requirement to save a monthly restore point, offsite, for 7 years. I was debating using the copy job (writing to the DR site) with monthly archive points, and then deduping them to save disk space, then it occurred to me, why bother with this, why can't i just use a monthly backup job that is the same selection list as my main job, and setting 84 (7 years) restore points?

The only advantage i can think of using the copy job is that i'll have multiple full backups, in case something ever happened to the main VBK.

Are there any other advantages i'm not considering, and if i was to do the monthly job, should i use forward incremental, with extreme compression ( i know dedupe may be pointless with this setting) or no compression and dedupe. Comments please :)
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

The only advantage i can think of using the copy job is that i'll have multiple full backups, in case something ever happened to the main VBK.
By default, backup copy job preserves one full backup and chain of dependent increments. If you want to have multiple full backups, you will have to specify GFS retention scheme.
If i was to do the monthly job, should i use forward incremental, with extreme compression ( i know dedupe may be pointless with this setting) or no compression and dedupe.
In order to guarantee max deduplcation rates, you should disable native VB&R compression/deduplication.

With monthly backup job you can probably stick to "always full backup" scenario, as the size of changes generated throughout the month might be approximately equal to the size of new full backup. Another advantage of "always full backup" scenario is that you will have multiple independent full backups.

Thanks.
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 »

to clarify, you mean i should uncheck "enable inline data deduplication" ?
namiko78
Expert
Posts: 117
Liked: 4 times
Joined: Mar 03, 2011 1:49 pm
Full Name: Steven Stirling
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by namiko78 »

and are you recommending creating a separate "monthly job" and using the active full method, or using the backup copy job with an every 30 day run time?
Thanks
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

You can leave inline deduplication as is, but disable compression and set LAN target as the storage optimization level.

It's up to chose either additional backup job or backup copy job. Both scenarios are popular among our customers. I was trying to say that if you finally stick to backup job, then, it will make sense to use "always full" approach, as this will give independent full backups, meanwhile, the additional amount of space needed will be negligible, comparing to monthly worth increments.

Thanks.
jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw »

Our Veeam backups were previously set to reversed incrementals, with no de-dup options selected - such as LAN target and compression disabled.

We have now enabled Dedup, however can anyone recommend what to do with existing backups? Are we better off archiving the old backups, and starting fresh with the recommended dedup settings to achieve maximum dedup results?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy »

Do you mean you have enabled Data Deduplication feature on the repository or just changed Veeam B&R job settings to the ones recommended when storing backups on a deduplicated volume?

Anyway, old backups will eventually fall out of the retention period and be deleted, so not a big deal. Do you have long chain of increments?
jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw »

foggy wrote:Do you mean you have enabled Data Deduplication feature on the repository or just changed Veeam B&R job settings to the ones recommended when storing backups on a deduplicated volume?

Anyway, old backups will eventually fall out of the retention period and be deleted, so not a big deal. Do you have long chain of increments?
We have enabled deduplication on a newly rebuilt server. Veeam was previously running on Server 2008 R2 - so we just rebuilt the server with 2012 R2, and re-attached our RDM disk. Installed the deduplication feature, and enabled. So far, we have only seen a 9% deduplication rate. These increments were previously written to the non-deduplicated disk with non-deduplication friendly settings.

I suppose over time, we should see an increased deduplication rate? I'd like to avoid re-starting backups.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

I suppose over time, we should see an increased deduplication rate?
The best deduplication rates will be achieved, if you switch to forward incremental backup mode. Thanks.
jpipdw
Influencer
Posts: 12
Liked: never
Joined: Mar 08, 2013 4:14 am
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jpipdw »

I have made this change in the backup job, and set it to Incremental. I have also set an Active Full to take place once a week. Looking forward to results!
VladV
Expert
Posts: 224
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

[MERGED] : Synology NAS iSCSI attached Enable 2012 Dedup

Post by VladV »

I am tinkering with the idea of enabling deduplication on the primary backup target which is a Synology DS1512+ attached to the Veeam server (2012 R2 VM) as an iSCSI target. Given that Microsoft states that their dedup has a minimal impact on performance wouldn't this be a good idea or am I missing something.

Regards,
Vlad
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

Hi, Vlad, your post has been merged into existing discussion about best practice of using MS deduplication on primary backup repository. Please, see answers provided above.

In general, it's still recommended to utilize dedupe appliances as secondary repository, not the primary one. If you're still going to use it as a primary repository, then, set appropriate dedupe settings ("Dedupe data older than") in order to guarantee fast access to recent backup data.

Thanks.
ChrisJ83
Influencer
Posts: 17
Liked: 1 time
Joined: Aug 11, 2014 7:44 am
Full Name: Chris Johnson
Contact:

[MERGED] : Server 2012 R2 Deduplication and Veeam deduplicat

Post by ChrisJ83 »

Hi

I have contacted Veeam support about this but I don't feel the support representative understood my query correctly so I thought I would see if anyone on the forum had any advice.

case 00620488

I am in the process of setting up a new backup server to run Veeam and it will run Server 2012 R2, I will then copy and archive backups onto a Synology SAN but I don't think that bit matters for this post.

I was thinking of enabling Server 2012 R2 deduplication on the backup volume and then keep inline data deduplication ticked however support seemed to insinuate I could only enable one or the other.

I personally thought Veeam would do its thing and then store the backup with its own inline deduplication and Windows would then provide another level of deduplication which works on small chunks of data. 4 full backups would pretty much become 1 full backup depending on the amount of changed data.

Any help/advie/experience with this appreciated.

Cheers
Chris
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Server 2012 R2 Deduplication and Veeam deduplication

Post by Gostev »

Hi! You can keep both enabled, in fact I would recommend that in terms of incremental backup performance. Having Veeam dedupe enabled will certainly not have any impact on Windows dedupe. Thanks!
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

Hi Chris,

Your post has been merged into existing discussion regarding similar matter, so please take a look at the answers provided above. In general, you can leave inline deduplication enabled, but disable compression for achieving better deduplication rates.

Thanks.
nunciate
Expert
Posts: 247
Liked: 39 times
Joined: May 21, 2013 9:08 pm
Full Name: Alan Wells
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by nunciate »

Here is my setup if you are interested in the numbers.
HP Server with SAS attached storage. 3 arrays with 12x 2Tb drives in RAID5 with 1 hotspare each.
In Windows they come up as 3 drives at about 18Tb each.
1 Drive has Deduplicaiton turned off. I have several very large backups files on that on of them is 9Tb big so I don't bother deduping those.
The other 2 drives have dedupe truned on and I get very good deduplication rates.
Drive 1 shows actual size of all files stored on the array at 37.6Tb. The disk shows only 15.9Tb used however.
Drive 2 shows actual size of all files stored on the array at 26.4Tb. The disk shows only 9.06Tb used.
These numbers don't reflect the deduplication that is still running from my weekend fulls so the space on disk will actully go up over the next day or 2 freeing up space for the next weekend fulls.

I can tell you that I started out using mostly Synthetic Full with reverse incremental. My deduplication settings are set process files older than 1 day. So the most recent files always get deduped.
At first reverse incrementals seemed to work fine. However, over time I noticed the weekly trnasforms started taking longer and longer. One backup with a vbk of less than 800Gb took 62 hours to process.
I since switched all of those backups to Incremental with a weekly active full. The numbers above reflect that setting and the backups run much faster now. The trick is to run the full backups so that you have time to deduplicate before filling up the drives. I run my fulls across several nights and all seems well.

In all I am backing up almost 50Tb of data across Thursday, Friday & Saturday nights. I am able to backup and tape out everything before Sunday evening with no issues.
gggggggg
Influencer
Posts: 17
Liked: never
Joined: Aug 26, 2014 1:50 am
Full Name: greg
Contact:

[MERGED] Windows v Veeam Deduplication

Post by gggggggg »

Hello, I am just planning our new Veeam install on Win 2012 R2.

To confirm I have this right:
[*]Veeam will de-duplicate at backup time, resulting in smaller VBK files
[*]Veeam will only de-duplicate servers within the same backup job
[*]Windows de-duplication will backup files on the disk, after Veeam has finished with the VBK

Sound right??

New question. In the job Advanced settings, Storage, Compression. DeDupe Friendly v others, will that have any impact on the deduplication ratios, or will it just use more RAM/CPU in the deduplication process?

thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy »

Greg, your understanding is correct. I've merged your post into existing discussion regarding Windows 2012 deduplication, please review for more details. Thanks.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by Shestakov »

Hello Greg,
Greg wrote:Veeam will de-duplicate at backup time, resulting in smaller VBK files
True. You can choose storage optimization options depending on the type of storage you select as a backup target.
Greg wrote:Veeam will only de-duplicate servers within the same backup job
Correct. Also deduplication eliminates blocks of free space, which decreases the size of the created backup file.
Greg wrote:Windows de-duplication will backup files on the disk, after Veeam has finished with the VBK
I`m a bit confused here. VBK is a backup file itself which will be stored on your repository after backup job is finished. Deduplication is a mechanism to make backup process more efficient, decrease .vbk file size.
Greg wrote:In the job Advanced settings, Storage, Compression. DeDupe Friendly v others, will that have any impact on the deduplication ratios, or will it just use more RAM/CPU in the deduplication process?
Yes, deduplication ratios depend on compression level. Also type of data on disks tampers deduplication and compression level.
Thanks.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by veremin »

nunciate wrote:At first reverse incrementals seemed to work fine. However, over time I noticed the weekly trnasforms started taking longer and longer. One backup with a vbk of less than 800Gb took 62 hours to process.
This is expected, as already deduped .vbk file has to be rehydrated in order for changes to be injected inside it. Thus, it's always recommended to put forward incremental mode with active full backup that doesn't touch any restore points and, thus, achieve better deduplication rates across multiple full backups. Thanks.
jadams159
Enthusiast
Posts: 80
Liked: 4 times
Joined: Apr 16, 2012 11:44 am
Full Name: Justin Adams
Location: United States
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by jadams159 »

Sightler, did you ever complete that whitepaper on Server 2012 dedupe you mentioned earlier in the thread?
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by Gostev »

AFAIK not... I think he hates it now, after seeing people struggle with one so much :D
pepijn
Influencer
Posts: 12
Liked: never
Joined: Jul 30, 2013 8:38 am
Full Name: Pepijn Deneut
Contact:

[MERGED] Windows server 2012 deduplication on backup reposit

Post by pepijn »

Hello,

I want to implement windows server 2012 deduplication on our backup repository.

Is this officially supported by Veeam?

I have seen the webinar on http://hyperv.veeam.com/winserver-dedup ... joep-2069/.

Are these still the best practices?

thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by foggy »

Totally supported. Please look through this thread for some hints regarding enabling dedupe on the repository.
pepijn
Influencer
Posts: 12
Liked: never
Joined: Jul 30, 2013 8:38 am
Full Name: Pepijn Deneut
Contact:

Re: Best Practice for MS Server 2012 DeDup Repo

Post by pepijn »

Thanks
Post Reply

Who is online

Users browsing this forum: BackItUp2020 and 249 guests