-
- Veteran
- Posts: 411
- Liked: 31 times
- Joined: Nov 21, 2014 10:05 pm
- Contact:
Re: v9: BitLooker Zeroing
Not sure if this is the case. Because the free space on the VM disk does not change THAT much. My guess is that dedup full optimization shuffles data around and rearrange it. That causes blocks to change and veeam to back them up. In the past I ran scheduled sdelete after on saturdays, and active full backup after that. It seemed to help. However, I removed that after v9 and bitlooker.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: v9: BitLooker Zeroing
Hi,
Absolutely correct.hyvokar wrote:My guess is that dedup full optimization shuffles data around and rearrange it. That causes blocks to change and veeam to back them up.
It is better to perform dedupe prior to backup. Otherwise the same blocks will be backed up twice: before deduplication and after deduplication due to CBT.hyvokar wrote:So can someone confirm, should bitlooker work with windows dedupe...
Please take a look at this post regarding bitlooker behaviour with reverse incremental mode.hyvokar wrote:...and forever reverse incremental backups ?
-
- Veteran
- Posts: 411
- Liked: 31 times
- Joined: Nov 21, 2014 10:05 pm
- Contact:
Re: v9: BitLooker Zeroing
Hi!PTide wrote:It is better to perform dedupe prior to backup. Otherwise the same blocks will be backed up twice: before deduplication and after deduplication due to CBT.
This is exactly what is happening. First windows dedup, then backup. I'm just wondering, why doesnt the bitlooker zeroing seem to have desider effects.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
-
- Veteran
- Posts: 411
- Liked: 31 times
- Joined: Nov 21, 2014 10:05 pm
- Contact:
Re: v9: BitLooker Zeroing
It just links back to this post.PTide wrote:Please take a look at this post regarding bitlooker behaviour with reverse incremental mode.
So my question still is: Should bitlooker work with Windows2012 deduplication and forever reverse incremental jobs, or do I need to continue using sdelete?
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: v9: BitLooker Zeroing
May I ask you if the results were the same without BitLooker enabled?This is exactly what is happening. First windows dedup, then backup. I'm just wondering, why doesnt the bitlooker zeroing seem to have desider effects.
Sdelete creates a huge file filled with zeroes and deletes it afterwards thus cleaning up space on your drive (including the deduped space). With reverse incremental that will cause backup job to place previous versions of all blocks that have been zeroed into .vrb.Should bitlooker work with Windows2012 deduplication and forever reverse incremental jobs, or do I need to continue using sdelete?
BitLooker detects if the file has been deleted and includes it into vrb.
Deduplication in Windows 2012 actually marks deduplicated data location as free thus BitLooker places that data into .vrb as well.
In other words - BitLooker does work with dedupe and reverse incremental chain in a strict accordance with all the rules I've descirbed. If you still feel that something is not working properly you can open a support case so our support team can take a closer look at your setup. Should you decide to open a case please post your case ID.
Thank you.
-
- Veteran
- Posts: 411
- Liked: 31 times
- Joined: Nov 21, 2014 10:05 pm
- Contact:
Re: v9: BitLooker Zeroing
So, I still need to use Sdelete to avoid backup files getting 200GB bigger due windows deduplication? I will open a support case, after the current ones are solved...
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
-
- Veteran
- Posts: 465
- Liked: 136 times
- Joined: Jul 16, 2015 1:31 pm
- Full Name: Marc K
- Contact:
Re: v9: BitLooker Zeroing
I personally don't think SDelete will help. SDelete is redundant if BitLooker is active. In my experience, I've found that Windows Dedup just generates changed blocks, sometimes a lot, as part of normal operation.
-
- Expert
- Posts: 224
- Liked: 25 times
- Joined: Apr 30, 2013 7:38 am
- Full Name: Vlad Valeriu Velciu
- Contact:
Re: v9: BitLooker Zeroing
And they will be picked up by the incremental or full backup when Bitlooker is disabled.
-
- Veteran
- Posts: 465
- Liked: 136 times
- Joined: Jul 16, 2015 1:31 pm
- Full Name: Marc K
- Contact:
Re: v9: BitLooker Zeroing
They'll get picked up even if BitLooker is enabled. The only way for them to not get picked up is if the files are created and deleted before the next backup runs. If you're doing daily backups, that's not likely to happen. When a garbage collection routine runs, it is working on blocks that have been around for more than a day.
-
- Veteran
- Posts: 411
- Liked: 31 times
- Joined: Nov 21, 2014 10:05 pm
- Contact:
Re: v9: BitLooker Zeroing
But in my case they seem to be picked up even if I have bitlooker enabledVladV wrote:And they will be picked up by the incremental or full backup when Bitlooker is disabled.
I will open support case on this one later.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
-
- Expert
- Posts: 224
- Liked: 25 times
- Joined: Apr 30, 2013 7:38 am
- Full Name: Vlad Valeriu Velciu
- Contact:
Re: v9: BitLooker Zeroing
Garbage collection deletes the references to deleted deduped files. After a deduplication run you will see the free space increase before GC thus the number of marked clean but dirty blocks increase also. Those blocks should not be picked up by Veeam when using Bitlooker as it is free space according to MFT.mkaec wrote:They'll get picked up even if BitLooker is enabled. The only way for them to not get picked up is if the files are created and deleted before the next backup runs. If you're doing daily backups, that's not likely to happen. When a garbage collection routine runs, it is working on blocks that have been around for more than a day.
-
- Veteran
- Posts: 465
- Liked: 136 times
- Joined: Jul 16, 2015 1:31 pm
- Full Name: Marc K
- Contact:
Re: v9: BitLooker Zeroing
When a file is deleted, its contents may be in a container that is shared with blocks from other files that haven't been deleted. Garbage collection goes through and repackages the blocks of the non-deleted files. This creates new/changed blocks that Veeam needs to back up.
-
- Expert
- Posts: 224
- Liked: 25 times
- Joined: Apr 30, 2013 7:38 am
- Full Name: Vlad Valeriu Velciu
- Contact:
Re: v9: BitLooker Zeroing
GC actually deletes (removes from MFT) the unreferenced chunks, it does not repackage anything. Repackaging comes when you defragment a dedup volume.mkaec wrote:When a file is deleted, its contents may be in a container that is shared with blocks from other files that haven't been deleted. Garbage collection goes through and repackages the blocks of the non-deleted files. This creates new/changed blocks that Veeam needs to back up.
If you monitor the dedup operation you will see that the free space increases (if it can find actual common blocks) by deleting redundant ones and creating smaller sized references. This deletion is considered free space by NTFS.
-
- Veteran
- Posts: 465
- Liked: 136 times
- Joined: Jul 16, 2015 1:31 pm
- Full Name: Marc K
- Contact:
Re: v9: BitLooker Zeroing
From https://support.microsoft.com/en-us/kb/3066175:
"Full garbage collection generates much more churn on the volume, because every chunk container is compacted (rewritten) if there are any unreferenced chunks."
"Full garbage collection generates much more churn on the volume, because every chunk container is compacted (rewritten) if there are any unreferenced chunks."
-
- Expert
- Posts: 224
- Liked: 25 times
- Joined: Apr 30, 2013 7:38 am
- Full Name: Vlad Valeriu Velciu
- Contact:
Re: v9: BitLooker Zeroing
"The data deduplication feature's garbage collection job reclaims the unreferenced chunks." from https://msdn.microsoft.com/en-us/librar ... s.85).aspx
The compaction that you referenced is made by deleting unreferenced chunks like I said. It also depends if you actual schedule a full GC or normal GC. Of course it has a very high churn because in reality a deletion is a modification and so a lot of blocks are modified. It may be that because CBT is referenced in 1MB blocks (I think) that actual free guest OS blocks aren't filling an entire CBT block so Veeam has to copy it.
The compaction that you referenced is made by deleting unreferenced chunks like I said. It also depends if you actual schedule a full GC or normal GC. Of course it has a very high churn because in reality a deletion is a modification and so a lot of blocks are modified. It may be that because CBT is referenced in 1MB blocks (I think) that actual free guest OS blocks aren't filling an entire CBT block so Veeam has to copy it.
-
- Veeam Software
- Posts: 712
- Liked: 168 times
- Joined: Nov 30, 2010 3:19 pm
- Full Name: Rick Vanover
- Location: Columbus, Ohio USA
- Contact:
Re: v9: BitLooker Zeroing
Does quick migration leverage the BitLooker logic? That would be awesome if it did. Since it is not configurable, I doubt it. I could test and see but thought I'd ask here.
-
- Chief Product Officer
- Posts: 31814
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: v9: BitLooker Zeroing
No, it does not.
-
- Veteran
- Posts: 465
- Liked: 136 times
- Joined: Jul 16, 2015 1:31 pm
- Full Name: Marc K
- Contact:
Re: v9: BitLooker Zeroing
I finally got around to testing this with some volumes that do not have Windows dedup active. The results were similar in that the dedup appliance did not like the first active full after BitLooker was turned on.PTide wrote:So I'm looking forward to hear about the results of a new test from you.Thank you!
Job 1
First Active Full After BitLooker - Veeam: 2130 GB; Appliance: 1417 GB
Second Active Full After BitLooker - Veeam: 2181 GB; Appliance 115 GB
Job 1
First Active Full After BitLooker - Veeam: 3550 GB; Appliance: 2180 GB
Second Active Full After BitLooker - Veeam: 3605 GB; Appliance 128GB
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: v9: BitLooker Zeroing
Thanks for sharing! First full dedupe ratio is poor, indeed, however ~19x and ~28x dedupe ratio on the second fulls looks good.
-
- Enthusiast
- Posts: 53
- Liked: 22 times
- Joined: May 10, 2010 7:21 am
- Full Name: Regi Pradeeswaran
- Contact:
[MERGED] Exclude deleted files tick box
Hi I found this useful information & I'm running the very latest version
https://3pardude.com/2016/06/29/getting ... version-9/
Is it OK to apply this setting ? Any advise
Cheers
https://3pardude.com/2016/06/29/getting ... version-9/
Is it OK to apply this setting ? Any advise
Cheers
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: v9: BitLooker Zeroing
Hi,
That depends on your environment, backup settings, and goals. Please review the thread and feel free to ask additional questions.
Thanks
That depends on your environment, backup settings, and goals. Please review the thread and feel free to ask additional questions.
Thanks
Who is online
Users browsing this forum: No registered users and 75 guests