Server 2012 Deduplication Notes/Workarounds

[dualdj1]

This is intended to be more informational, as I have found a few quirks with 2012 that may be useful to know, if you choose to use Server 2012 as your deduplication device.

These few points may or may not have any direct effect, when strictly using VEEAM; I found these more as I am using another software still for tape archiving (Backup Exec).


1) When using deduplication, a high volume of disk traffic is generated. As such, any applications employing VSS for backing up files can experience problems, depending on the size of your disk. For example, one of my drives is aproximately 30tb total, with around 10tb used. During the time to run my tape backup job to backup my vbk files, almost 500gb of data was generated by VSS. This produced vss errors in the windows system log similar to the following:

Source: Volsnap
EventID: 25
The shadow copies of volume G: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.

I'm not sure that this would cause any problems with VEEAM directly (if it doesn't use VSS for tape/backup copy jobs, should be ok), but any other systems utilizing vss to copy your backup files could be effected. The solution is to use vssadmin to add shadowstorage on a different volume. I'm using a dedicated volume for shadowcopy storage.

vssadmin add shadowstorage /for=G: /on=e: /maxsize=25% (maxsize is based on source volume, not target volume)

to view allocated shadowstorage:

vssadmin list shadowstorage


2) Deduplication can cause the archive bit to be set on VEEAM backup files. As part of the deduplication process, it appears that the files are modified (as would be expected), and as a result the archive bit is then set, even though the "view" of the file does not change. This makes it difficult to use incremental backups using archive bit. You would need to use scripts, or other methods, to make sure the archive bit is reset on the files which don't need backed up prior to starting your job, or use a different method for identifying changed files.


3) Changes to the Optimization schedule will not hold, unless you manually create the optimization jobs (don't let windows autocreate the job).


I'm sure there are plenty more, and didn't see a topic really like this. Feel free to merge this with a better topic, or Add additional notes relating to using Server 2012 deduplication.

[tntteam]

Hi there and thanks for your advices

I'm adding my 2 cents :

I was using 2 deduplication schedules, 1 per drive, each with /memory 70 which was kind of dumb but the process were never consuming the memory, I was always like 150GB consumed / 192 GB total. But I noticed I was having big disk activity on C: -> it was using the pagefile.sys or in other words, swapping.
Reduced /memory to 50 for each job and now deduplication works faster and no more activity on C: drive. Each job consume approx 45GB of RAM out of 192GB total.

My jobs are now :

%systemroot%\system32\ddpcli.exe enqueue /opt /scheduled /vol V: /priority high /throttle none /memory 50
%systemroot%\system32\ddpcli.exe enqueue /opt /scheduled /vol D: /priority high /throttle none /memory 50

Also I increased the window size of deduplication scheduled task, I end up with less performance for backups (as deduplication eats IOPS) but it's acceptable.

For info, here are the stats of the dedup :

FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
30.12 TB 92.92 TB 58 74 V:
19.95 TB 72.01 TB 40 51 D:

Each backup copy file is between 1TB - 3 TB depending on the job.