Copying 2012r2 dedupe data only

[bbricker]

This is admittedly not a VB&R ONLY specific question, but I know you guys have dealt with this topic a lot. Is there a way to copy (or replicate) a Windows 2012r2 deduplicated volume to another 2012r2 server without it rehydrating everything first? Ideally I want a copy/replicate method where the process only looks at the changed data from the last 24 hours and copies only that deduped portion.

[Gostev]

Is it a physical or virtual server?

[bbricker]

Virtual. And just to try and anticipate what you might be about to ask, ha, I have tried using VB&R to replicate it, but I guess it was seeing way more changed blocks than actual changed dedupe space within the VM, because it could never keep up with the data being changed each day.

I'm adding about 1.2TB new data per day to this 2012r2 vm and after Windows dedupe processes have done their magic, it only grows about 20GB a day (it's a big repo server for SQL/DB2 backup dump files, so very little data actually changing). Veeam replication job was seeing and trying to move closer to 1TB each day.

[bbricker]

Anyone have any ideas? I've tried Windows DFSR but it turns out it is not made for files this large. DFSR works great on small files like in a typical file server role, but it chokes on the size of files I'm storing (half TB). Surely someone working with VB&R with multiple 2012r2 dedupe servers has tackled this

[foggy]

I'm adding about 1.2TB new data per day to this 2012r2 vm and after Windows dedupe processes have done their magic, it only grows about 20GB a day (it's a big repo server for SQL/DB2 backup dump files, so very little data actually changing). Veeam replication job was seeing and trying to move closer to 1TB each day.

That's expected, since Windows deduplication seriously affects the number of changed blocks inside VM and Veeam B&R image-level processing has to copy much more than the estimated data change rate inside guest OS.

[tsightler]

Plus, it's post-process, so if you write 1TB of backup data to the machine that 1TB of change happens immediately and, as you seem to be aware, only after that point does Windows perform it's dedupe magic. As a block level backup product with no knowledge of the actual on-disk change, Veeam will just replicate it all.

Interestingly, when researching an issue for a client a few weeks ago, I ran across this product that claims the ability to replicate Windows 2012R2 dedupe volumes without rehydration and the website specifically talked about replicating a Veeam repository, so it would seem like it should work for SQL dumps as well. Note that this is not an endorsement of the product as I really know nothing about it other than a random Google search leading me there, but I thought you might find it interesting given your current requirement.

[bbricker]

Thanks Tom, that sounds like it's exactly what I want! I'm just a little disappointed it costs $5000 The thing is, I have a gigabit connection between my DC and DR site, so I really have plenty of bandwidth and then some for all my DR needs. For example, before I got our VB&R updated recently so I could implement the G-F-S Copy feature (which, is AWESOME, thank you guys), I was using plain robocopy to push 5-6TB of VBK/VIBs each week/night to my DR site and not having any problem fitting the time windows. So I guess I can afford to fat-copy my 1.2TB of SQL dump data each night to an identical 2012r2 dedupe server at the DR site and just let it get re-deduped. It's just that I like for things to be efficient for efficiency sake. Not so much for $5000 though. Oh well!

edit: ha, on the flip side, I guess if I spent $5K on some dedupe friendly replicating software, maybe I wouldn't need a gigabit connection to the DR site I will say though, it is awfully nice having all that bandwidth for various purposes. And I haven't yet started doing full production vm-replication to the DR site yet, that is still in the works. Continuing to evaluate different methods/products.