Comprehensive data protection for all workloads
Post Reply
mcclans
Influencer
Posts: 16
Liked: 2 times
Joined: Feb 17, 2015 2:05 pm
Full Name: Shawn McClanahan
Contact:

v8 - Estimating full backup size with Windows dedup

Post by mcclans »

What would the expected outcome be in the size of your backup files after enabling Windows dedup. It seems like even though we're saving 860GB in deduplicated data within Windows we're not realizing those space savings within the size of the backup. When you look at the settings for this vm from within the job itself, the calculated size is listed as 1.9TB even though we're only using rougly 900GB. The job was created prior to enabling deduplication. I don't have an active case open for this issue but we have other file servers that are planned to enable dedup but we're now wondering if we're missing something. Just wanted to get the experience of others.
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by Gostev »

mcclans wrote:even though we're only using rougly 900GB
What specifically do you mean by this?
mcclans
Influencer
Posts: 16
Liked: 2 times
Joined: Feb 17, 2015 2:05 pm
Full Name: Shawn McClanahan
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by mcclans »

Hi Gostev,

Prior to enabling dedupication the volume had 1.9 TB of used space. After dedup the volume had 900 GB of used space so the size of the vm was cut almost in half. It didn't seem like the backup shrunk the same percentage though.
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by Gostev »

OK, I understand your scenario now, thanks for explaining.

From NTFS perspective within guest OS, you now have 1TB of free disk space available. However, remember that the way NTFS works, deleting files does change the actual disk contents (all file data remains on this disk, which is why undelete is possible). Correspondingly, the backup size will not change either - because from block-level processing perspective, VM image still contains 1.9 GB worth of "dirty" blocks which need to be backed up. You can search this forum for "sdelete" for more information on how you can "fix" this.

Thanks!
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

[MERGED] w2012r2 backup size

Post by hyvokar »

Hi!

I have a win2012r2 server with two disks, disk1 100GB (20GB free) and disk2 1.3TB (0.7TB Free).
When I take a backup from this machine, the resulting backup size is 1.2TB. I have set dedup for LAN target and compression on optimal size.
I have set Windows dedup on disk2, and rate is about 50%.

So my question is, shouldn't the backup size be siginificantly lower (around 700GB even without dedup and compression) and how can I achieve it?
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by foggy »

Please review considerations above for an answer.
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by hyvokar » 1 person likes this post

Hi and thank you for your reply.

I tried the sdelete -z on my other fileserver for both volumes (since sdelete is running very slow on first one..).
Win2012r2, disk1 100BGB(50GB Free), disk2 1TB(500GB Free), the resulting vkb file is still way over 800GB in size. .vrb is 300GB, so obiously sdelete did something :) Do I need to make active full backup or take some other steps to reduze the full backup file size?
I'm doing reverse incremental backups.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by veremin »

hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by hyvokar »

Hi!
Didnt move the VM. Active full did the trick.
Thanks!
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by veremin »

You're welcome. Should other questions arise, don't hesitate to let us know.
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by hyvokar »

v.Eremin wrote:You're welcome. Should other questions arise, don't hesitate to let us know.
Sdelete finished on the first sever also, ran full active backup and backup size went down 700GB as expected.
Just out of curiosity, what should moving the VM to another datastore afted sdelete do ?
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by foggy »

In case you run sdelete against a thin disk, it is expanded to the maximum provisioned size and you need to do Storage vMotion to reclaim space.
cffit
Veteran
Posts: 338
Liked: 35 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

[MERGED] Backup bigger after running Windows 2012 Dedup

Post by cffit »

We have a file server that was around 925GB in size. The VEEAM full backup file would be around 750GB.

I ran the Windows 2012 File Server Deduplication option on the server and now the server is around 690GB. However, the VEEAM full backup file got bigger rather than smaller. It is 870GB. That's larger than the space showing as used on the VM.

Does anyone have an idea of why this would be?
cffit
Veteran
Posts: 338
Liked: 35 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by cffit »

Is it best to do a defrag of the local VM before running sdelete so that the most blocks are zeroed out as possible? Is this kind of thing good maintenance to do once in a while on all VMs to keep their backup size down?
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by hyvokar »

cffit wrote:Is it best to do a defrag of the local VM before running sdelete so that the most blocks are zeroed out as possible? Is this kind of thing good maintenance to do once in a while on all VMs to keep their backup size down?
Would not recommended defragging VMs. It will move data around and your backups will get larger. Also, it will grow your thin provisioned disks (if you are using any).
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: [MERGED] Backup bigger after running Windows 2012 Dedup

Post by hyvokar »

cffit wrote:We have a file server that was around 925GB in size. The VEEAM full backup file would be around 750GB.

I ran the Windows 2012 File Server Deduplication option on the server and now the server is around 690GB. However, the VEEAM full backup file got bigger rather than smaller. It is 870GB. That's larger than the space showing as used on the VM.

Does anyone have an idea of why this would be?
Hi!

You should run sdelete -z on your volumes and then take active full backup.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
cupplesey
Enthusiast
Posts: 50
Liked: 3 times
Joined: Aug 06, 2015 8:08 am
Full Name: Andy C
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by cupplesey »

hyvokar wrote: Would not recommended defragging VMs. It will move data around and your backups will get larger. Also, it will grow your thin provisioned disks (if you are using any).
In addtion, if you have storage snapshots such as Netapp then this will cause your snaps will also grow considerably (not taking into account the larger Veeam backups) depending on the amount zeroed blocks on that given volume/LUN/NFS and this will be seen as changes. Therefore overall the volume(s)/aggregate storage could grow and can create a big knock on effect especially if the SAN/storage is getting close to capacity.
lsolty
Novice
Posts: 7
Liked: 1 time
Joined: Jul 18, 2013 7:01 am
Contact:

[MERGED]: Backup Size doubled after deduplication inside<...

Post by lsolty »

Hello everyone,

we are using VEEAM B&R 8.0.0.917 running on a Windows Server 2012 (physical machine). Our file server is a virtualized (vSphere 5.5) Windows Server 2012. We have about 900GB of data on the file server. The veeam backup size of this VM was about half of this size in the past.

Recently we activated the Windows Server 2012 build-in deduplication on the file server and saved about 44% of space inside the VM. Suprisingly, the size of the veeam backup doubled then. So we saved space inside the VM and lost disk space on the veeam server. I expected the veeam backup size not to change and I am wondering, what happened.

I deleted the veeam job and created a new one. This had no impact on the backup size, it is still double the size than before the activation of Server 2012 deduplication inside the VM.

Can anyone explain?

Thanks,
lsolty
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by PTide »

Hi,

Your topic has been merged with an existing one, please see the thread.

Thank you.
hyvokar
Veteran
Posts: 406
Liked: 29 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by hyvokar »

Hi!

Windows dedup seems to be a bit more of a problem than I originally thought. After couple of months I'm back in over 1TB backup size.

This the backup size bumps up 100-200GB once a month. This is most likely due full garbage collection that happens every fourth time.
Now, I could disable full garbage collectiong, but I'm not too keen to do that.
A little more complicated way of resolving this is to schedule sdelete after a full gc and then schedule a active full backup. However, will this affect my backup copy job (read: after a active full backup, will I need to retransfer all the data in next backup copy job and if now, should the backup copy job size be about the same than the active full?) ?
Any other ideas how to tackle this problem?
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by foggy »

Backup copy job will transfer only changes, regardless of the fact that the source job had active full before that.
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by lando_uk »

Hi, I noticed on the prerequisites for restoring VM Guest OS files (Windows) there's a part about 2012 R2 Dedupe VMs.

"If you plan to back up VMs running Microsoft Windows Server 2012 R2 and Data Deduplication is enabled for some VM volumes, you must deploy Veeam Backup & Replication on a machine running Microsoft Windows Server 2012 R2 with Data Deduplication feature enabled. Otherwise, the restore process may fail."

Does this mean the main VBR server needs to be on 2012R2, or just the repository where the VM jobs are stored?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by foggy »

This means that Veeam B&R itself (backup server) must be installed on 2012R2. This, however, is no longer a requirement for v9, where you can specify a particular mount server for FLR.
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by lando_uk »

Would the mount server have to receive the whole VBK file or would it just extract the relevant VM files?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by dellock6 »

The mount server will act like the VBR server does today. It will have to "mount" the entire vbk (and the other incrementals depending on the restore point) to read the content, but the transfert to the original or new location will be limited to the required amount of data.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by albertwt »

Hi Vladimir,

So in order to achieve the smalles VM backup job, do I have to Storage vMotion the deduped volume AND then followed by the manual Active Full backup after the deduplication process completed ?
--
/* Veeam software enthusiast user & supporter ! */
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v8 - Estimating full backup size with Windows dedup

Post by foggy »

The move is required in case of thin disks only, since they are expanded to the maximum provisioned size after running sdelete against them.
Post Reply

Who is online

Users browsing this forum: FelixW, MartinO, Regnor and 157 guests