Host-based backup of VMware vSphere VMs.
Post Reply
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Backing up Windows 2012 dedupe server

Post by readie » 1 person likes this post

We have two fairly similar Windows fileservers, both about 3+ TB, so our major time consumers for full backups etc.
I have noticed recently that one ALWAYS seems to take much longer than the other for backups, and replication from backups. They are both in the same jobs, so it is difficult to analyse details unless I am watching when it happens, or check later.
The one which takes longer is Windows 2012 R2 with Windows DeDupe switched on. Does this perhaps mean that Windows is regularly messing around deduping the disks and producing changes which then trigger CBT changes although there have been no actual file changes? Both servers have about the same quantity of user changes to files.
Any thoughts or recommendations?
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backing up Windows 2012 dedupe server

Post by veremin » 1 person likes this post

Does this perhaps mean that Windows is regularly messing around deduping the disks and producing changes which then trigger CBT changes although there have been no actual file changes?
Your understanding is perfectly correct. Usage of Windows Deduplication (with garbage collection being the main culprit) inside the guest leads to high number of changed blocks. Thus, longer backup time. Thanks.
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: Backing up Windows 2012 dedupe server

Post by readie »

Thanks - do you happen to know whether it is possible to schedule Windows to only 'work' its dedupe perhaps once a month . . . and accept that we would then have a slower backup once a month? Now it has done the bulk of its dedupe, and we are saving around 25% disk space, we don't really need it to work every day or even every week. (I realise this is a Windows question rather than Veeam - but thought I would ask.)
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backing up Windows 2012 dedupe server

Post by foggy »

I don't think you can do that via UI, but I believe you can schedule a script doing this monthly.
pkelly_sts
Veteran
Posts: 600
Liked: 66 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

[MERGED] Impact of 2012 de-dupe on changed blocks?

Post by pkelly_sts » 1 person likes this post

I'm in the planning stages of moving a 1Tb (900Gb used) data volume on a 2008 server to a new 2012 R2 server and am planning on enabling dedupe on that server some time after migration.

The only thing that concerns me is the amount of block changes that enabling dedupe might cause, thus creating an abnormally large daily incremental which we might struggle to copy off-site over the 20Mb link we currently have.

So, firstly, does enabling in-VM dedupe create a large incremental at all and, if so, is it recommended to simply configure the dedupe config with a very old age of files to start deduping and gradually reduce the age over time (e.g. only dedupe any files older than, say, 3yrs initially, then after that has gone over the wire, reduce to older than 2.5yrs, then 2, then 1.5 etc.)

Any thoughts recommendations appreciated!

NB: I'm working on a different plan to actually move the drive over whilst not causing a 900Gb daily change in itself, currently hoping to effectively replace the C: drive of the original VM with the C: drive of the newly built 2012 VM, thus the C: drive is the only change to that VM so incremental relatively small. I can then rename things as appropriately afterwards & use sVmotion to get the underlying files renamed.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: [MERGED] Impact of 2012 de-dupe on changed blocks?

Post by foggy » 1 person likes this post

pkelly_sts wrote:So, firstly, does enabling in-VM dedupe create a large incremental at all
Yes.
pkelly_sts wrote:is it recommended to simply configure the dedupe config with a very old age of files to start deduping and gradually reduce the age over time (e.g. only dedupe any files older than, say, 3yrs initially, then after that has gone over the wire, reduce to older than 2.5yrs, then 2, then 1.5 etc.)
This will not help in the long run, since such tasks as periodic garbage collection will still make the enormous amount of changes on the disk with deduplication enabled.
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Backing up Windows 2012 dedupe server

Post by albertwt »

v.Eremin wrote: Your understanding is perfectly correct. Usage of Windows Deduplication (with garbage collection being the main culprit) inside the guest leads to high number of changed blocks. Thus, longer backup time. Thanks.
So is there any suggested best practice settings to optimize the Windows Server 2012 R2 deduplicated file server Veeam Backup ? :x
--
/* Veeam software enthusiast user & supporter ! */
YoMarK
Enthusiast
Posts: 55
Liked: 8 times
Joined: Jul 13, 2009 12:50 pm
Full Name: Mark
Location: The Netherlands
Contact:

Re: Backing up Windows 2012 dedupe server

Post by YoMarK » 1 person likes this post

My recommendation is that you should not use Windows 2012R2 deduplication for general purpose file servers anyway.
dedup is incompatible with certain windows features like Windows Search(can not index deduped files), but that may or may not be an issue for you.

When you still want to use deduplication, you "sort of" NEED to run garbage collection to reclaim unused space. You can schedule this manually, but sooner of later it WILL result in a lot of changed blocks in the VMDK and that WILL result in longer backup times.
There is probably no way around it.
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: Backing up Windows 2012 dedupe server

Post by skrause » 1 person likes this post

If you check ignore deleted file blocks in the advanced job settings, it cuts way down on the amount of changed blocks when a garbage collection happens in my experience. (There will still be some as the chunklets in windows dedupe and the blocks Veeam reads don't always align).

Is it the amount of time the job takes to run or the amount of backup storage used? If you are concerned about disk space usage, set the garbage collection job in task scheduler to run on the day that your full happens (even if it is synthetic). That will solve the "large increments" problem.
Steve Krause
Veeam Certified Architect
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Backing up Windows 2012 dedupe server

Post by albertwt »

skrause,

Thanks for the sharing and tips.
I never run Full backup on my Veeam Backup job since it is running OK with the Reverse Incremental job.

Or shall I convert the Dedupe backup job into Forever Forward Incremental to get better result in disk space saving instead ?
--
/* Veeam software enthusiast user & supporter ! */
pkelly_sts
Veteran
Posts: 600
Liked: 66 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: Backing up Windows 2012 dedupe server

Post by pkelly_sts » 1 person likes this post

Apologies for my lack of clarity, in my case I'm talking about the *initial* enabling of de-dupe.

We actually already have another file server with de-dupe enabled that causes us no problems at all, but this was a newly built server at the time which had data gradually moved to it over a period of time whereas my current server is a pre-existing VM with the 900Gb of data.

We currently get around 45% de-dupe out of the existing server so it's a valuable space-saver for us!
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: Backing up Windows 2012 dedupe server

Post by skrause »

albertwt wrote:skrause,

Thanks for the sharing and tips.
I never run Full backup on my Veeam Backup job since it is running OK with the Reverse Incremental job.

Or shall I convert the Dedupe backup job into Forever Forward Incremental to get better result in disk space saving instead ?
Forever Forward and Reverse Incremental should give you pretty the same disk space usage. The difference is when the transform happens and how long your snapshot is open on the VM being backed up.

If you aren't doing a periodic full operation on a VM running windows dedupe you are probably never going to be able to avoid the "large increment" problem as you can't cover up the changed blocks from the garbage collection operation with the full.

Running dedupe on production servers really comes down to this for me:
Will it save me a substantial amount of expensive (production) disk space?
and
Am I willing/able to offset the disk space saved on the expensive disk with additional less expensive (backup) disk space to cover the higher change rate?

In any situation where a VM has an extremely high change rate, from my experience it makes the most sense to run an active full once a week rather than doing transforms because the transform operations take much longer and use more resources than just taking the full does.
Steve Krause
Veeam Certified Architect
Jean-Philippe
Influencer
Posts: 12
Liked: 2 times
Joined: Mar 20, 2017 9:26 am
Location: Lyon

[MERGED] Backup and Replicate Windows 2016 VM with deduplica

Post by Jean-Philippe »

Hi, I would like to know if they are some best practices arround backup and replication of Windows 2016 (works also for 2012) Virtual Machine running Deduplication at the guest level ?

What's going on with the CBT ?
I mean that even if at the guest level files are not changing a lot does the deduplication process modify the signature of the blocks and so the CBT.
If the CBT change then Veeam should take the blocks to backup or replicate them and hence increase time and amout of data transfered to repository ?

Thanks in advance for your help,
regards.

Jean-Philippe
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Backup and Replicate Windows 2016 VM with deduplication

Post by Mike Resseler »

Hi Jean-Philippe,

*Edit: Welcome to the forums!

So if I understand it correctly, you have a VM and inside that VM you have a data volume which uses windows server deduplication. Correct?

For CBT: Only the changed blocks are transferred. The way windows dedupe works is that (before deduplication) the data is written somewhere and after that the blocks are moved to the chunk store. Then they are not going to be changed again (additional pointers might be created to them but that is in the metadata) unless you rehydrate.

So it might be that you have sometimes bigger transfer depending on your dedupe settings (For example, daily backup and dedupe setting of not deduping a file that is younger then 3 days. After 3 days it gets deduped and those will be seen as new blocks)

Makes sense?
Jean-Philippe
Influencer
Posts: 12
Liked: 2 times
Joined: Mar 20, 2017 9:26 am
Location: Lyon

Re: Backup and Replicate Windows 2016 VM with deduplication

Post by Jean-Philippe »

Hello MIke, thank you for your reply.

That's the point : VM Windows 2016 (File server) with the Deduplication feature enabled.
My concern is more about replication, since the dedup have been activated time to replicate the Vm from one hot to another have increase by 10.

That's why I was asking about block changing and CBT.
I'm going to open a case but I would like first to check if they are some best pratices between Veeam and "Guest Dedup"

Regards,
Jean-Philippe
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup and Replicate Windows 2016 VM with deduplication

Post by foggy »

Hi Jean-Philippe, as Mike's mentioned, in-guest dedupe does generate a lot of changes inside VM, affecting both backup and replication jobs. Please review the thread above for some other considerations.
Jean-Philippe
Influencer
Posts: 12
Liked: 2 times
Joined: Mar 20, 2017 9:26 am
Location: Lyon

Re: Backing up Windows 2012 dedupe server

Post by Jean-Philippe »

Ok thank you Foggy.
briggs
Enthusiast
Posts: 76
Liked: 3 times
Joined: Mar 22, 2013 11:01 am
Full Name: Alan Thompson
Contact:

[MERGED] Handling Win2012R2 dedup VM's

Post by briggs »

All,
How can I manage the backing up of a Windows 2012 R2 VM that has de-dup enabled?

To elaborate a little. Our main file/print server is a 2012 R2 VM with 2TB disk space. This space fills up so quickly nowadays, that I couldn't avoid enabling Dedup. As soon as I've done that, whenever the dedup cycle/schedule (once a week, Friday evening), the resulting incremental is massive. Nearly the size of the full.

I simply can't handle backups this large, with the retention period I have (70 restore points, backups every 12 hours Mon-Sat). Basically, now it's enabled, once a week I get a single 1-2TB incremental where it was once around 100GB.

An even bigger problem is that we use Veeam Cloud connect, and getting a backup file that size out the door and into the cloud is very tricky in the window we have, and the VCC service obviously charges by the GB so we are running up huge bills unless I sort it out ASAP.

- The onsite job is an incremental with monthly fulls
- The VCC job is a reverse incremental

I do realise that dedup effectively changes the disk blocks, and understand that's the way it is, but I'm after an sort of recommendation or advice on how people deal with this problem.

Thanks in advance, Alan
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backing up Windows 2012 dedupe server

Post by PTide »

Alan,

Unfortunately there is not much what can be done at the moment. You should consider increasing the size of the disk and disabling in-guest dedupe.

Thanks
Post Reply

Who is online

Users browsing this forum: anpa, Semrush [Bot], uszy and 82 guests