Best Practice for MS Server 2012 DeDup Repo

Availability for the Always-On Enterprise

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Delo123 » Mon Sep 12, 2016 7:35 am

If I understood correctly, Windows 2016 will even be worse for 1TB+ files. Currently 1TB+ files are being deduped but it takes quite long which isn't that much of an issue anymore due to Veeam's scale out repositories. As incrementals now can go to different volumes the dedupe job on the fulls doesn't have to be interrupted. However in Server 2016 i read that windows dedupe will only dedupe the first TB so for a 8TB file, 1TB will be deduped, 7 will not :( That's why will still hope Veeam will come with an option to spilt vbk's in smaller chunks...
Delo123
Expert
 
Posts: 299
Liked: 82 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby lightsout » Mon Sep 12, 2016 1:42 pm

I've not heard that, but I guess we will find out!
lightsout
Expert
 
Posts: 180
Liked: 46 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Gostev » Mon Sep 12, 2016 4:57 pm

Delo123 wrote:However in Server 2016 i read that windows dedupe will only dedupe the first TB so for a 8TB file

First almost 4TB actually, although we did not re-test on RTM yet. That 1TB is the true limit for modified files, however backup files are never touched once created (at least with the recommended job settings for deduplicating storage).
Gostev
VP, Product Management
 
Posts: 20960
Liked: 2239 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Seve CH » Tue Sep 27, 2016 12:47 pm

Hi

I have not read the documentation for W2016, but dedup seems not to be supported on W2012R2 with volumes larger than 64TB. It is a VSS issue rather than a dedup problem. Windows will let you activate deduplication and won't warn you, but it will produce all kind of weird errors, mostly of the kind of "The parameter is incorrect."

https://support.microsoft.com/en-us/kb/2967756

Regards.
Seve CH
Influencer
 
Posts: 10
Liked: 6 times
Joined: Mon May 09, 2016 2:34 pm
Location: Switzerland
Full Name: JM Severino

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby Delo123 » Wed Sep 28, 2016 6:32 am

Hi Steve, that is correct :) for that reason most of create a storage pool and carve multiple thin volumes from that each just under 64TB. Backups and even better scale out repositories splitting up fulls and incrementals can de distributed among these giving us multiple dedupe threads etc. :)
Delo123
Expert
 
Posts: 299
Liked: 82 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Best Practice for MS Server 2012 DeDup Repo

Veeam Logoby cheese » Wed Feb 08, 2017 1:18 pm

Hello

Just to save someone else from any pain when formatting very large volumes.

You can use the /q switch combined with /L

format g: /A:64k /q returns (as expecteded):

C:\Users\administrator>fsutil fsinfo ntfsinfo g:
NTFS Volume Serial Number : 0x0422992622991e2c
NTFS Version : 3.1
LFS Version : 2.0
Number Sectors : 0x0000001b491defff
Total Clusters : 0x0000000036923bdf
Free Clusters : 0x0000000036922a0e
Total Reserved : 0x0000000000000040
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 65536
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x0000000000010000
Mft Start Lcn : 0x000000000000c000
Mft2 Start Lcn : 0x0000000000000001
Mft Zone Start : 0x000000000000c000
Mft Zone End : 0x000000000000cca0
Max Device Trim Extent Count : 0
Max Device Trim Byte Count : 0x0
Max Volume Trim Extent Count : 62
Max Volume Trim Byte Count : 0x40000000
Resource Manager Identifier : A4A80A8E-ED80-11E6-90E7-00259069050F

However using the /q switch as well:

C:\Users\administrator>format g: /A:64k /L /q
The type of the file system is NTFS.
Enter current volume label for drive G: Veeam Extent 2

WARNING, ALL DATA ON NON-REMOVABLE DISK
DRIVE G: WILL BE LOST!
Proceed with Format (Y/N)? y
QuickFormatting 54.6 TB
Volume label (32 characters, ENTER for none)? Veeam Extent 2
Creating file system structures.
Format complete.
54.6 TB total disk space.
54.6 TB are available.

C:\Users\administrator>fsutil fsinfo ntfsinfo g:
NTFS Volume Serial Number : 0x98c4dbf1c4dbcf9c
NTFS Version : 3.1
LFS Version : 2.0
Number Sectors : 0x0000001b491defff
Total Clusters : 0x0000000036923bdf
Free Clusters : 0x0000000036922a01
Total Reserved : 0x0000000000000040
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 65536
Bytes Per FileRecord Segment : 4096
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x0000000000100000
Mft Start Lcn : 0x000000000000c000
Mft2 Start Lcn : 0x0000000000000001
Mft Zone Start : 0x000000000000c000
Mft Zone End : 0x000000000000cca0
Max Device Trim Extent Count : 0
Max Device Trim Byte Count : 0x0
Max Volume Trim Extent Count : 62
Max Volume Trim Byte Count : 0x40000000
Resource Manager Identifier : A4A80A99-ED80-11E6-90E7-00259069050F


That's saved my several days formatting 2x 54TB drives - hope someone finds it helpful.
cheese
Lurker
 
Posts: 1
Liked: never
Joined: Wed Feb 08, 2017 9:04 am
Full Name: Dave

Previous

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot] and 7 guests