Comprehensive data protection for all workloads
Post Reply
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Copying 2012r2 dedupe data only

Post by bbricker »

This is admittedly not a VB&R ONLY specific question, but I know you guys have dealt with this topic a lot. Is there a way to copy (or replicate) a Windows 2012r2 deduplicated volume to another 2012r2 server without it rehydrating everything first? Ideally I want a copy/replicate method where the process only looks at the changed data from the last 24 hours and copies only that deduped portion.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Copying 2012r2 dedupe data only

Post by Gostev »

Is it a physical or virtual server?
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Re: Copying 2012r2 dedupe data only

Post by bbricker »

Virtual. And just to try and anticipate what you might be about to ask, ha, I have tried using VB&R to replicate it, but I guess it was seeing way more changed blocks than actual changed dedupe space within the VM, because it could never keep up with the data being changed each day.

I'm adding about 1.2TB new data per day to this 2012r2 vm and after Windows dedupe processes have done their magic, it only grows about 20GB a day (it's a big repo server for SQL/DB2 backup dump files, so very little data actually changing). Veeam replication job was seeing and trying to move closer to 1TB each day.
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Re: Copying 2012r2 dedupe data only

Post by bbricker »

Anyone have any ideas? I've tried Windows DFSR but it turns out it is not made for files this large. DFSR works great on small files like in a typical file server role, but it chokes on the size of files I'm storing (half TB). Surely someone working with VB&R with multiple 2012r2 dedupe servers has tackled this :-)
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Copying 2012r2 dedupe data only

Post by foggy »

bbricker wrote:I'm adding about 1.2TB new data per day to this 2012r2 vm and after Windows dedupe processes have done their magic, it only grows about 20GB a day (it's a big repo server for SQL/DB2 backup dump files, so very little data actually changing). Veeam replication job was seeing and trying to move closer to 1TB each day.
That's expected, since Windows deduplication seriously affects the number of changed blocks inside VM and Veeam B&R image-level processing has to copy much more than the estimated data change rate inside guest OS.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Copying 2012r2 dedupe data only

Post by tsightler »

Plus, it's post-process, so if you write 1TB of backup data to the machine that 1TB of change happens immediately and, as you seem to be aware, only after that point does Windows perform it's dedupe magic. As a block level backup product with no knowledge of the actual on-disk change, Veeam will just replicate it all.

Interestingly, when researching an issue for a client a few weeks ago, I ran across this product that claims the ability to replicate Windows 2012R2 dedupe volumes without rehydration and the website specifically talked about replicating a Veeam repository, so it would seem like it should work for SQL dumps as well. Note that this is not an endorsement of the product as I really know nothing about it other than a random Google search leading me there, but I thought you might find it interesting given your current requirement.
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Re: Copying 2012r2 dedupe data only

Post by bbricker »

Thanks Tom, that sounds like it's exactly what I want! I'm just a little disappointed it costs $5000 :( The thing is, I have a gigabit connection between my DC and DR site, so I really have plenty of bandwidth and then some for all my DR needs. For example, before I got our VB&R updated recently so I could implement the G-F-S Copy feature (which, is AWESOME, thank you guys), I was using plain robocopy to push 5-6TB of VBK/VIBs each week/night to my DR site and not having any problem fitting the time windows. So I guess I can afford to fat-copy my 1.2TB of SQL dump data each night to an identical 2012r2 dedupe server at the DR site and just let it get re-deduped. It's just that I like for things to be efficient for efficiency sake. Not so much for $5000 though. Oh well!

edit: ha, on the flip side, I guess if I spent $5K on some dedupe friendly replicating software, maybe I wouldn't need a gigabit connection to the DR site :wink: I will say though, it is awfully nice having all that bandwidth for various purposes. And I haven't yet started doing full production vm-replication to the DR site yet, that is still in the works. Continuing to evaluate different methods/products.
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Semrush [Bot] and 265 guests