Recommended settings when using a dedupe appliance

Availability for the Modern Data Center

Recommended settings when using a dedupe appliance

Veeam Logoby markwyeth » Wed Mar 28, 2012 10:30 pm

We are testing a the FalconStor dedupe appliance and interested to know what is recommended when using these type of devices. Specifically:

1. Should inline data deduplication be enabled or disabled
2. Assuming we turn off compression
3. Do we set storage optimization to: Local, LAN or WAN

Cheers Mark
markwyeth
Novice
 
Posts: 8
Liked: never
Joined: Fri Oct 01, 2010 1:38 am
Full Name: Mark Wyeth

Re: Recommended settings when using a dedupe appliance

Veeam Logoby tfloor » Thu Mar 29, 2012 6:46 am

We always have used:

1. Dedup enabled
2. Compression definitly off!
3. Storage to LAN.

You can play with option 3, because it depends also on the VM's you're backing up.
If you for example backup an oracle vm with much disk read/writes then you can set it to WAN if you think you're incrementals are too big.
But if it's a CIFS/SMB dedup appliance i recommend LAN

Cheers,

Tristan.
tfloor
Expert
 
Posts: 266
Liked: 14 times
Joined: Tue Jan 03, 2012 2:02 pm
Full Name: Tristan Floor

Re: Recommended settings when using a dedupe appliance

Veeam Logoby dellock6 » Thu Mar 29, 2012 1:39 pm

Uhm, not sure about the inline deduplication.
To date , the only appliance where both Veeam and the dedup vendor stated it can be left enabled is ExaGrid. Tristan, which dedup appliance are you using?
Luca Dell'Oca
EMEA Evangelist @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 2210
Liked: 418 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Recommended settings when using a dedupe appliance

Veeam Logoby Gostev » Thu Mar 29, 2012 9:00 pm

So far, I have not seen a dedupe appliance where overall resulting data reduction ratio would suffer from keeping Veeam deduplication on. Remember that Veeam dedupe is done with large block size. At the same time, keeping Veeam dedupe on may speed up processing significantly.

Consider using Low compression as well, besides None. Low compression algorithm does not mess up advanced dedupe, but may speed up processing significantly.
Gostev
Veeam Software
 
Posts: 16175
Liked: 944 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: Recommended settings when using a dedupe appliance

Veeam Logoby dellock6 » Thu Mar 29, 2012 10:43 pm

After several tests with ExaGrid, I found out the best combination is inline deduplication on and compression off.
But dedup appliances are really different one another, so everyone of them need best practices and vendor advice to be used at its best.
Do not know the official position by FalconStor.
Luca Dell'Oca
EMEA Evangelist @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 2210
Liked: 418 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Recommended settings when using a dedupe appliance

Veeam Logoby tsightler » Fri Mar 30, 2012 1:36 am 1 person likes this post

Inline deduplication on and compression off is pretty much the answer for every deduplication solution, however, some vendors focus heavily on wanting to show "maximum" dedupe ratio, so they suggest turning off Veeam dedupe. If your criteria for using a dedupe appliance is that you want the largest "dedupe ratio" possible, then of course you want to turn off Veeam dedupe. However, in most cases, the better goal is to focus on storing and transferring the least amount of data, as that will give the best performance, and in some cases, even be willing to give up a little dedupe/storage for performance.

To illustrate why some vendors like to recommend disabling Veeam deduplication consider this example. Let's say you are backing up a disk that has 40 identical 256K blocks, that's 10MB of identical data. Now lets assume that one of those blocks can be compressed and deduped 4:1 for a 75% savings. If you leave Veeam dedupe on Veeam will write only 1 of those 40 blocks, since the others are identical, that's a 40:1 savings in network traffic to the dedupe appliance, but the dedupe appliance will only report 4:1 dedupe. If we disable Veeam dedupe, Veeam will copy all 40 of the blocks to the dedupe appliance, thus copying 10MB instead of 256K, however, the dedupe appliance will deduplicate the 40 blocks, and report a dedupe ratio of 160:1. The amount of data on disk remained the same, but the amount transferred across the wire is 40x more. Once again, I recognize the contrived aspect of this example, but I'm only illustrating the point.

Using Veeam low compression can be a nice performance improvement, saving 10-20% in the total amount of data that must transfer across the wire, while increasing the size on disk typically by less the 5% do to slightly poorer dedupe. This 10-20% savings across the wire can really add up to some improved performance if you have many TB of full backups to run. Dedupe appliances can only ingest a fixed amount of TB's an hour and thus this performance improvement can translate into shorter backups, and faster restores due to less data having to be transferred across the wire during restores as well, but at the penalty of some additional loss of storage dedupe.

As with everything, it's all about striking the right balance for your environment.
tsightler
Veeam Software
 
Posts: 3572
Liked: 858 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Recommended settings when using a dedupe appliance

Veeam Logoby tfloor » Fri Mar 30, 2012 5:21 am

dellock6 wrote:Uhm, not sure about the inline deduplication.
To date , the only appliance where both Veeam and the dedup vendor stated it can be left enabled is ExaGrid. Tristan, which dedup appliance are you using?

I WAS using a datadomain. DD610. Too slow. So I've placed it in /dev/null. :)
tfloor
Expert
 
Posts: 266
Liked: 14 times
Joined: Tue Jan 03, 2012 2:02 pm
Full Name: Tristan Floor

Re: Recommended settings when using a dedupe appliance

Veeam Logoby stormlight » Sun Apr 01, 2012 3:50 pm

I can't remember where I read this, or maybe watched it on a Veeam webinar:

Go to your Dedup appliance in Veeam under the backup repositories. Go to the advance tab where location is

Enable Align backup file data blocks
Enable Decompress backup data blocks before storing.

These two have helped me with my dedup box. DataDomain.
stormlight
Enthusiast
 
Posts: 34
Liked: 1 time
Joined: Thu Apr 28, 2011 5:34 pm
Full Name: JG

Re: Recommended settings when using a dedupe appliance

Veeam Logoby Gostev » Sun Apr 01, 2012 6:20 pm

Definitely avoid enabling backup file data blocks align for deduplication storage that uses variable block size dedupe (such as DataDomain). What saved you here is that this checkbox does not currently work until you add one missing registry key ;)
Gostev
Veeam Software
 
Posts: 16175
Liked: 944 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: Recommended settings when using a dedupe appliance

Veeam Logoby stormlight » Sun Apr 01, 2012 6:33 pm

What you are saying is in conflict with this webinar or maybe the webinar was to generic and did not define the importance of storage that uses variable block size deDup. (I would hope that the presenter would outline that next time)

In addition the webinar makes no mention of the registry key.

http://go.veeam.com/webinar-12202011-ri ... am-ty.html

Is the decompress data blocks setting ok to check? Does this help with restores at all ?

Edit: I see the webinar was based on exagrid.
stormlight
Enthusiast
 
Posts: 34
Liked: 1 time
Joined: Thu Apr 28, 2011 5:34 pm
Full Name: JG

Re: Recommended settings when using a dedupe appliance

Veeam Logoby Gostev » Sun Apr 01, 2012 7:32 pm

Yes, not using compression is always a good idea with any deduplicating storage.
Gostev
Veeam Software
 
Posts: 16175
Liked: 944 times
Joined: Sun Jan 01, 2006 1:01 am
Full Name: Anton Gostev

Re: Recommended settings when using a dedupe appliance

Veeam Logoby dellock6 » Mon Apr 02, 2012 6:41 pm

As an addition for those using ExaGrid, you can ignore block align since ExaGrid uses variable size, compression is needed only if you have the ExaGrid off-site, so you're best compressing the backup in the local proxy and then de-compress at the target site before saving data to ExaGrid.
Uhm, I'm better write a blog post about these settings so I will not need to copy/paste them in so many threads :)

Luca.
Luca Dell'Oca
EMEA Evangelist @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 2210
Liked: 418 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Veean and Deduplication Appliances

Veeam Logoby bengoa@tulane.edu » Wed May 30, 2012 9:29 pm

[merged]

Is any one out there using a Deduplication Appliances with their VEEAM backups?

What appliance are you using? What kind of compression ratio are you seeing?

Are you backing up raw/ or are you using VEEAM deduping option as well?

We are looking at the DELL DR4000 backup appliance. Any insight would be appreciated.

Thanks,
Alex
bengoa@tulane.edu
Novice
 
Posts: 7
Liked: never
Joined: Wed Sep 21, 2011 7:55 pm
Full Name: Alex Bengoa

Re: Veean and Deduplication Appliances

Veeam Logoby pcmmitham » Wed May 30, 2012 10:52 pm

we're using steelheads for deduplication, nothing is better than them for that. but interacting with veeam....we're still working on it. its seemless for everyother app we use, but I think it might be whats causing veeam to fail miserably at replication. once we can replicate successfully I'll update this thread!!
pcmmitham
Member
 
Posts: 18
Liked: never
Joined: Tue Jan 17, 2012 6:18 pm
Full Name: Pierre Mitham

Re: Veean and Deduplication Appliances

Veeam Logoby tsightler » Thu May 31, 2012 12:07 am

@pcmmitham - Are you referring to WAN deduplication? I think the OP was referring to deduplication storage appliances. If you are attempting to use Veeam replication with Steelheads than you'll want to disable multiple TCP connections as well as Veeam compression. Should work great. I have quite a few clients using the combination successfully.
tsightler
Veeam Software
 
Posts: 3572
Liked: 858 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: averylarry, Bing [Bot] and 15 guests