Backing up Windows 2012 dedupe server

VMware specific discussions

Backing up Windows 2012 dedupe server

Veeam Logoby readie » Sat Apr 30, 2016 5:14 pm 1 person likes this post

We have two fairly similar Windows fileservers, both about 3+ TB, so our major time consumers for full backups etc.
I have noticed recently that one ALWAYS seems to take much longer than the other for backups, and replication from backups. They are both in the same jobs, so it is difficult to analyse details unless I am watching when it happens, or check later.
The one which takes longer is Windows 2012 R2 with Windows DeDupe switched on. Does this perhaps mean that Windows is regularly messing around deduping the disks and producing changes which then trigger CBT changes although there have been no actual file changes? Both servers have about the same quantity of user changes to files.
Any thoughts or recommendations?
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Backing up Windows 2012 dedupe server

Veeam Logoby v.Eremin » Mon May 02, 2016 8:43 am 1 person likes this post

Does this perhaps mean that Windows is regularly messing around deduping the disks and producing changes which then trigger CBT changes although there have been no actual file changes?

Your understanding is perfectly correct. Usage of Windows Deduplication (with garbage collection being the main culprit) inside the guest leads to high number of changed blocks. Thus, longer backup time. Thanks.
v.Eremin
Veeam Software
 
Posts: 13290
Liked: 971 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Backing up Windows 2012 dedupe server

Veeam Logoby readie » Tue May 03, 2016 7:33 am

Thanks - do you happen to know whether it is possible to schedule Windows to only 'work' its dedupe perhaps once a month . . . and accept that we would then have a slower backup once a month? Now it has done the bulk of its dedupe, and we are saving around 25% disk space, we don't really need it to work every day or even every week. (I realise this is a Windows question rather than Veeam - but thought I would ask.)
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Backing up Windows 2012 dedupe server

Veeam Logoby foggy » Tue May 03, 2016 8:17 am

I don't think you can do that via UI, but I believe you can schedule a script doing this monthly.
foggy
Veeam Software
 
Posts: 14746
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] Impact of 2012 de-dupe on changed blocks?

Veeam Logoby pkelly_sts » Thu Oct 20, 2016 8:57 am 1 person likes this post

I'm in the planning stages of moving a 1Tb (900Gb used) data volume on a 2008 server to a new 2012 R2 server and am planning on enabling dedupe on that server some time after migration.

The only thing that concerns me is the amount of block changes that enabling dedupe might cause, thus creating an abnormally large daily incremental which we might struggle to copy off-site over the 20Mb link we currently have.

So, firstly, does enabling in-VM dedupe create a large incremental at all and, if so, is it recommended to simply configure the dedupe config with a very old age of files to start deduping and gradually reduce the age over time (e.g. only dedupe any files older than, say, 3yrs initially, then after that has gone over the wire, reduce to older than 2.5yrs, then 2, then 1.5 etc.)

Any thoughts recommendations appreciated!

NB: I'm working on a different plan to actually move the drive over whilst not causing a 900Gb daily change in itself, currently hoping to effectively replace the C: drive of the original VM with the C: drive of the newly built 2012 VM, thus the C: drive is the only change to that VM so incremental relatively small. I can then rename things as appropriately afterwards & use sVmotion to get the underlying files renamed.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: [MERGED] Impact of 2012 de-dupe on changed blocks?

Veeam Logoby foggy » Thu Oct 20, 2016 12:10 pm 1 person likes this post

pkelly_sts wrote:So, firstly, does enabling in-VM dedupe create a large incremental at all

Yes.

pkelly_sts wrote:is it recommended to simply configure the dedupe config with a very old age of files to start deduping and gradually reduce the age over time (e.g. only dedupe any files older than, say, 3yrs initially, then after that has gone over the wire, reduce to older than 2.5yrs, then 2, then 1.5 etc.)

This will not help in the long run, since such tasks as periodic garbage collection will still make the enormous amount of changes on the disk with deduplication enabled.
foggy
Veeam Software
 
Posts: 14746
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backing up Windows 2012 dedupe server

Veeam Logoby albertwt » Thu Oct 20, 2016 12:22 pm

v.Eremin wrote:Your understanding is perfectly correct. Usage of Windows Deduplication (with garbage collection being the main culprit) inside the guest leads to high number of changed blocks. Thus, longer backup time. Thanks.


So is there any suggested best practice settings to optimize the Windows Server 2012 R2 deduplicated file server Veeam Backup ? :x
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Backing up Windows 2012 dedupe server

Veeam Logoby YoMarK » Fri Oct 21, 2016 2:00 pm 1 person likes this post

My recommendation is that you should not use Windows 2012R2 deduplication for general purpose file servers anyway.
dedup is incompatible with certain windows features like Windows Search(can not index deduped files), but that may or may not be an issue for you.

When you still want to use deduplication, you "sort of" NEED to run garbage collection to reclaim unused space. You can schedule this manually, but sooner of later it WILL result in a lot of changed blocks in the VMDK and that WILL result in longer backup times.
There is probably no way around it.
YoMarK
Enthusiast
 
Posts: 35
Liked: 2 times
Joined: Mon Jul 13, 2009 12:50 pm
Location: The Netherlands
Full Name: Mark

Re: Backing up Windows 2012 dedupe server

Veeam Logoby skrause » Fri Oct 21, 2016 4:02 pm 1 person likes this post

If you check ignore deleted file blocks in the advanced job settings, it cuts way down on the amount of changed blocks when a garbage collection happens in my experience. (There will still be some as the chunklets in windows dedupe and the blocks Veeam reads don't always align).

Is it the amount of time the job takes to run or the amount of backup storage used? If you are concerned about disk space usage, set the garbage collection job in task scheduler to run on the day that your full happens (even if it is synthetic). That will solve the "large increments" problem.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: Backing up Windows 2012 dedupe server

Veeam Logoby albertwt » Fri Oct 21, 2016 10:35 pm

skrause,

Thanks for the sharing and tips.
I never run Full backup on my Veeam Backup job since it is running OK with the Reverse Incremental job.

Or shall I convert the Dedupe backup job into Forever Forward Incremental to get better result in disk space saving instead ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Backing up Windows 2012 dedupe server

Veeam Logoby pkelly_sts » Mon Oct 24, 2016 10:05 am 1 person likes this post

Apologies for my lack of clarity, in my case I'm talking about the *initial* enabling of de-dupe.

We actually already have another file server with de-dupe enabled that causes us no problems at all, but this was a newly built server at the time which had data gradually moved to it over a period of time whereas my current server is a pre-existing VM with the 900Gb of data.

We currently get around 45% de-dupe out of the existing server so it's a valuable space-saver for us!
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: Backing up Windows 2012 dedupe server

Veeam Logoby skrause » Tue Oct 25, 2016 2:43 pm

albertwt wrote:skrause,

Thanks for the sharing and tips.
I never run Full backup on my Veeam Backup job since it is running OK with the Reverse Incremental job.

Or shall I convert the Dedupe backup job into Forever Forward Incremental to get better result in disk space saving instead ?


Forever Forward and Reverse Incremental should give you pretty the same disk space usage. The difference is when the transform happens and how long your snapshot is open on the VM being backed up.

If you aren't doing a periodic full operation on a VM running windows dedupe you are probably never going to be able to avoid the "large increment" problem as you can't cover up the changed blocks from the garbage collection operation with the full.

Running dedupe on production servers really comes down to this for me:
Will it save me a substantial amount of expensive (production) disk space?
and
Am I willing/able to offset the disk space saved on the expensive disk with additional less expensive (backup) disk space to cover the higher change rate?

In any situation where a VM has an extremely high change rate, from my experience it makes the most sense to run an active full once a week rather than doing transforms because the transform operations take much longer and use more resources than just taking the full does.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

[MERGED] Backup and Replicate Windows 2016 VM with deduplica

Veeam Logoby Jean-Philippe » Mon Mar 20, 2017 9:41 am

Hi, I would like to know if they are some best practices arround backup and replication of Windows 2016 (works also for 2012) Virtual Machine running Deduplication at the guest level ?

What's going on with the CBT ?
I mean that even if at the guest level files are not changing a lot does the deduplication process modify the signature of the blocks and so the CBT.
If the CBT change then Veeam should take the blocks to backup or replicate them and hence increase time and amout of data transfered to repository ?

Thanks in advance for your help,
regards.

Jean-Philippe
Jean-Philippe
Novice
 
Posts: 5
Liked: 1 time
Joined: Mon Mar 20, 2017 9:26 am

Re: Backup and Replicate Windows 2016 VM with deduplication

Veeam Logoby Mike Resseler » Mon Mar 20, 2017 9:49 am

Hi Jean-Philippe,

*Edit: Welcome to the forums!

So if I understand it correctly, you have a VM and inside that VM you have a data volume which uses windows server deduplication. Correct?

For CBT: Only the changed blocks are transferred. The way windows dedupe works is that (before deduplication) the data is written somewhere and after that the blocks are moved to the chunk store. Then they are not going to be changed again (additional pointers might be created to them but that is in the metadata) unless you rehydrate.

So it might be that you have sometimes bigger transfer depending on your dedupe settings (For example, daily backup and dedupe setting of not deduping a file that is younger then 3 days. After 3 days it gets deduped and those will be seen as new blocks)

Makes sense?
Mike Resseler
Veeam Software
 
Posts: 3165
Liked: 362 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Backup and Replicate Windows 2016 VM with deduplication

Veeam Logoby Jean-Philippe » Mon Mar 20, 2017 10:04 am

Hello MIke, thank you for your reply.

That's the point : VM Windows 2016 (File server) with the Deduplication feature enabled.
My concern is more about replication, since the dedup have been activated time to replicate the Vm from one hot to another have increase by 10.

That's why I was asking about block changing and CBT.
I'm going to open a case but I would like first to check if they are some best pratices between Veeam and "Guest Dedup"

Regards,
Jean-Philippe
Jean-Philippe
Novice
 
Posts: 5
Liked: 1 time
Joined: Mon Mar 20, 2017 9:26 am

Next

Return to VMware vSphere



Who is online

Users browsing this forum: erbr, Google [Bot] and 31 guests