Comprehensive data protection for all workloads
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v9: BitLooker Zeroing

Post by hyvokar » 2 people like this post

Not sure if this is the case. Because the free space on the VM disk does not change THAT much. My guess is that dedup full optimization shuffles data around and rearrange it. That causes blocks to change and veeam to back them up. In the past I ran scheduled sdelete after on saturdays, and active full backup after that. It seemed to help. However, I removed that after v9 and bitlooker.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
PTide
Product Manager
Posts: 6551
Liked: 765 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: v9: BitLooker Zeroing

Post by PTide »

Hi,
hyvokar wrote:My guess is that dedup full optimization shuffles data around and rearrange it. That causes blocks to change and veeam to back them up.
Absolutely correct.
hyvokar wrote:So can someone confirm, should bitlooker work with windows dedupe...
It is better to perform dedupe prior to backup. Otherwise the same blocks will be backed up twice: before deduplication and after deduplication due to CBT.
hyvokar wrote:...and forever reverse incremental backups ?
Please take a look at this post regarding bitlooker behaviour with reverse incremental mode.
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v9: BitLooker Zeroing

Post by hyvokar »

PTide wrote:It is better to perform dedupe prior to backup. Otherwise the same blocks will be backed up twice: before deduplication and after deduplication due to CBT.
Hi!

This is exactly what is happening. First windows dedup, then backup. I'm just wondering, why doesnt the bitlooker zeroing seem to have desider effects.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v9: BitLooker Zeroing

Post by hyvokar »

PTide wrote:Please take a look at this post regarding bitlooker behaviour with reverse incremental mode.
It just links back to this post.


So my question still is: Should bitlooker work with Windows2012 deduplication and forever reverse incremental jobs, or do I need to continue using sdelete?
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
PTide
Product Manager
Posts: 6551
Liked: 765 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: v9: BitLooker Zeroing

Post by PTide »

This is exactly what is happening. First windows dedup, then backup. I'm just wondering, why doesnt the bitlooker zeroing seem to have desider effects.
May I ask you if the results were the same without BitLooker enabled?
Should bitlooker work with Windows2012 deduplication and forever reverse incremental jobs, or do I need to continue using sdelete?
Sdelete creates a huge file filled with zeroes and deletes it afterwards thus cleaning up space on your drive (including the deduped space). With reverse incremental that will cause backup job to place previous versions of all blocks that have been zeroed into .vrb.

BitLooker detects if the file has been deleted and includes it into vrb.

Deduplication in Windows 2012 actually marks deduplicated data location as free thus BitLooker places that data into .vrb as well.

In other words - BitLooker does work with dedupe and reverse incremental chain in a strict accordance with all the rules I've descirbed. If you still feel that something is not working properly you can open a support case so our support team can take a closer look at your setup. Should you decide to open a case please post your case ID.

Thank you.
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v9: BitLooker Zeroing

Post by hyvokar »

So, I still need to use Sdelete to avoid backup files getting 200GB bigger due windows deduplication? I will open a support case, after the current ones are solved...
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
mkaec
Veteran
Posts: 465
Liked: 136 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9: BitLooker Zeroing

Post by mkaec »

I personally don't think SDelete will help. SDelete is redundant if BitLooker is active. In my experience, I've found that Windows Dedup just generates changed blocks, sometimes a lot, as part of normal operation.
VladV
Expert
Posts: 224
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

Re: v9: BitLooker Zeroing

Post by VladV »

And they will be picked up by the incremental or full backup when Bitlooker is disabled.
mkaec
Veteran
Posts: 465
Liked: 136 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9: BitLooker Zeroing

Post by mkaec »

They'll get picked up even if BitLooker is enabled. The only way for them to not get picked up is if the files are created and deleted before the next backup runs. If you're doing daily backups, that's not likely to happen. When a garbage collection routine runs, it is working on blocks that have been around for more than a day.
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: v9: BitLooker Zeroing

Post by hyvokar »

VladV wrote:And they will be picked up by the incremental or full backup when Bitlooker is disabled.
But in my case they seem to be picked up even if I have bitlooker enabled :(

I will open support case on this one later.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
VladV
Expert
Posts: 224
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

Re: v9: BitLooker Zeroing

Post by VladV »

mkaec wrote:They'll get picked up even if BitLooker is enabled. The only way for them to not get picked up is if the files are created and deleted before the next backup runs. If you're doing daily backups, that's not likely to happen. When a garbage collection routine runs, it is working on blocks that have been around for more than a day.
Garbage collection deletes the references to deleted deduped files. After a deduplication run you will see the free space increase before GC thus the number of marked clean but dirty blocks increase also. Those blocks should not be picked up by Veeam when using Bitlooker as it is free space according to MFT.
mkaec
Veteran
Posts: 465
Liked: 136 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9: BitLooker Zeroing

Post by mkaec »

When a file is deleted, its contents may be in a container that is shared with blocks from other files that haven't been deleted. Garbage collection goes through and repackages the blocks of the non-deleted files. This creates new/changed blocks that Veeam needs to back up.
VladV
Expert
Posts: 224
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

Re: v9: BitLooker Zeroing

Post by VladV »

mkaec wrote:When a file is deleted, its contents may be in a container that is shared with blocks from other files that haven't been deleted. Garbage collection goes through and repackages the blocks of the non-deleted files. This creates new/changed blocks that Veeam needs to back up.
GC actually deletes (removes from MFT) the unreferenced chunks, it does not repackage anything. Repackaging comes when you defragment a dedup volume.

If you monitor the dedup operation you will see that the free space increases (if it can find actual common blocks) by deleting redundant ones and creating smaller sized references. This deletion is considered free space by NTFS.
mkaec
Veteran
Posts: 465
Liked: 136 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9: BitLooker Zeroing

Post by mkaec »

From https://support.microsoft.com/en-us/kb/3066175:

"Full garbage collection generates much more churn on the volume, because every chunk container is compacted (rewritten) if there are any unreferenced chunks."
VladV
Expert
Posts: 224
Liked: 25 times
Joined: Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu
Contact:

Re: v9: BitLooker Zeroing

Post by VladV »

"The data deduplication feature's garbage collection job reclaims the unreferenced chunks." from https://msdn.microsoft.com/en-us/librar ... s.85).aspx

The compaction that you referenced is made by deleting unreferenced chunks like I said. It also depends if you actual schedule a full GC or normal GC. Of course it has a very high churn because in reality a deletion is a modification and so a lot of blocks are modified. It may be that because CBT is referenced in 1MB blocks (I think) that actual free guest OS blocks aren't filling an entire CBT block so Veeam has to copy it.
Rick.Vanover
Veeam Software
Posts: 712
Liked: 168 times
Joined: Nov 30, 2010 3:19 pm
Full Name: Rick Vanover
Location: Columbus, Ohio USA
Contact:

Re: v9: BitLooker Zeroing

Post by Rick.Vanover » 1 person likes this post

Does quick migration leverage the BitLooker logic? That would be awesome if it did. Since it is not configurable, I doubt it. I could test and see but thought I'd ask here.
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v9: BitLooker Zeroing

Post by Gostev »

No, it does not.
mkaec
Veteran
Posts: 465
Liked: 136 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9: BitLooker Zeroing

Post by mkaec »

PTide wrote:So I'm looking forward to hear about the results of a new test from you.Thank you!
I finally got around to testing this with some volumes that do not have Windows dedup active. The results were similar in that the dedup appliance did not like the first active full after BitLooker was turned on.

Job 1
First Active Full After BitLooker - Veeam: 2130 GB; Appliance: 1417 GB
Second Active Full After BitLooker - Veeam: 2181 GB; Appliance 115 GB

Job 1
First Active Full After BitLooker - Veeam: 3550 GB; Appliance: 2180 GB
Second Active Full After BitLooker - Veeam: 3605 GB; Appliance 128GB
PTide
Product Manager
Posts: 6551
Liked: 765 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: v9: BitLooker Zeroing

Post by PTide »

Thanks for sharing! First full dedupe ratio is poor, indeed, however ~19x and ~28x dedupe ratio on the second fulls looks good.
regi
Enthusiast
Posts: 53
Liked: 22 times
Joined: May 10, 2010 7:21 am
Full Name: Regi Pradeeswaran
Contact:

[MERGED] Exclude deleted files tick box

Post by regi »

Hi I found this useful information & I'm running the very latest version

https://3pardude.com/2016/06/29/getting ... version-9/

Is it OK to apply this setting ? Any advise

Cheers
PTide
Product Manager
Posts: 6551
Liked: 765 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: v9: BitLooker Zeroing

Post by PTide »

Hi,

That depends on your environment, backup settings, and goals. Please review the thread and feel free to ask additional questions.

Thanks
Post Reply

Who is online

Users browsing this forum: No registered users and 75 guests