Windows Deduplication Repository; SS vs Hardware Raid

Availability for the Always-On Enterprise

Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby drnash21 » Mon Mar 06, 2017 4:05 pm

We have been using a Dell R720XD+MD1200 Windows 2012R2 server with deduplication for a backup repository for some time with success. Unfortunately as the amount of systems we are backing up has grown we have outgrown the capacity and performance of the current repository. We have purchased another enclosure and more disks to expand capacity and i wanted to address the raid setup to fix the performance issues. Currently we are using the H710+H810 raid cards in Raid 6. Unfortunately the write speed of this isn't great obviously. This is causing dedup jobs to run 2-3 days and backup windows are starting to be missed without all backups completing. I was planning on moving to a LSI 9207-8i/e hba with Windows storage spaces setup with a pci ssd as cache to improve the performance of the repository.

Before doing so could anyone share there experience in the performance of storage spaces vs hardware raid as a deduplication backup repositiory?
drnash21
Lurker
 
Posts: 2
Liked: never
Joined: Thu Jul 05, 2012 9:47 pm
Full Name: Daniel Nash

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby nmdange » Mon Mar 06, 2017 10:09 pm

I can't speak to Parity Storage Spaces performance, I've only used Mirror spaces. I've heard that parity performance is pretty terrible. With Storage Spaces Direct, it's supposed to be better, but that is a very different setup hardware wise.

I use RAID 50 for my backup repositories and I get good performance. The write performance with RAID 50 is going to be better than RAID 6. You can still sustain multiple drive failures with RAID 50, but it depends on which drives fail.

Also you might be better of moving to Windows Server 2016 and using ReFS. Actually deduplication is also a lot faster in WS 2016 too, as long as they fix the data corruption issue ;)
nmdange
Expert
 
Posts: 217
Liked: 59 times
Joined: Thu Aug 20, 2015 9:30 pm

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby drnash21 » Mon Mar 06, 2017 10:50 pm

Thanks for the input. I have the license and was planning on moving to 2016 as well once the data corruption issue was fixed. My understanding was that dedup didn't work on ReFS. Has that changed?
drnash21
Lurker
 
Posts: 2
Liked: never
Joined: Thu Jul 05, 2012 9:47 pm
Full Name: Daniel Nash

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby Gostev » Mon Mar 06, 2017 10:59 pm

I can confirm parity storage spaces performance is pretty bad, I would not recommend it for primary backup repository - but rather for secondary one (archival) repositories only.

Dedupe does not work on ReFS, but fast cloning prevents duplication in the first place (aka spaceless full backups).
Gostev
Veeam Software
 
Posts: 21512
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby Mike Resseler » Tue Mar 07, 2017 6:19 am

I can only agree with Gostev around the ReFS story. It is a great story but MSFT has some fixes to make. But since I am a Monday morning reader of Gostev's digest :-) I have read that MSFT is making progress and getting things fixed so fingers crossed.

For deduplication, if you would decide not to go for S2D and ReFS but want to go back to deduplication in server 2016, there are a lot of differences there also. You can change priorities of the dedup process, assign cores, amount of memory so tweaking it to your needs has really improved and when you have a beefy system you can go far now. It still comes with downsides (and now needs fixes also, something MSFT is also working on) such as the TB limit per file but I would certainly not write it off yet and do some testing with it

My 2 cents
M/
Mike Resseler
Veeam Software
 
Posts: 3346
Liked: 380 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby DonZoomik » Tue Mar 07, 2017 8:06 am

I believe SS Parity would have usable (not great performance) in right conditions. I'm planning a large backup repository based on a SuperMicro 60-disk chassis with 6 NVMe SSDs.
My plan:
* 64TB volumes with maximum column size (17)
* Increasing chunk size (512K, 1M...)
* SSD write cache with 100GB Write Cache per volume (6*2TB SSD with high DWPD such as Intel P3700)
* Setting IsPowerProtected flag to allow relaxed caching and flushing. While it increases corruption risk on power failure, I'd say its a reasonable compromise with dual power rails with independent UPS and a generator.
With 10TB disks and a few SSDs I'd expect total write throughput of at least 3GB/s (conservative) over 10 or so volumes. While not great, it's reasonable and likely to choke SANs and FC/10G. And price-performance-density-power ratio would be pretty good.

Some guy did some testing here http://vault-tec.info/post/129360770011 ... sis-part-1
With some tuning... it could work but only at big scales with you start to max out networking.
DonZoomik
Enthusiast
 
Posts: 25
Liked: 8 times
Joined: Fri Nov 25, 2016 1:56 pm

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby Gostev » Tue Mar 07, 2017 3:20 pm

Please do let us know how it goes! We're all ears for sure :D
Gostev
Veeam Software
 
Posts: 21512
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby skrause » Tue Mar 07, 2017 7:14 pm

One way to alleviate the long deduplication runs is to use smaller volumes in Windows. Windows dedupe is single threaded per windows volume.

Of course, it is a bit messy with internal drives on the same RAID controller but if you are in a situation where currently you have say, one 36TB "E:" disk in Windows, you separate it into three 12TB volumes "E:" "F:" and "G:" your machine will be able to be more efficient on the CPU/RAM utilization. The amount of reads/writes by the controller might still be an issue and dealing with more repositories in Veeam is a bit more fiddly, but you might find better performance.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 307
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: Windows Deduplication Repository; SS vs Hardware Raid

Veeam Logoby Gostev » Tue Mar 07, 2017 7:25 pm

skrause wrote:Windows dedupe is single threaded per windows volume.

This is no longer the case with Windows Server 2016.
Gostev
Veeam Software
 
Posts: 21512
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: KeiichiKun, ncapponi, opg70, PParkin, Schulti and 82 guests