Recommended settings when using a dedupe appliance

[markwyeth]

We are testing a the FalconStor dedupe appliance and interested to know what is recommended when using these type of devices. Specifically:

1. Should inline data deduplication be enabled or disabled
2. Assuming we turn off compression
3. Do we set storage optimization to: Local, LAN or WAN

Cheers Mark

[tfloor]

We always have used:

1. Dedup enabled
2. Compression definitly off!
3. Storage to LAN.

You can play with option 3, because it depends also on the VM's you're backing up.
If you for example backup an oracle vm with much disk read/writes then you can set it to WAN if you think you're incrementals are too big.
But if it's a CIFS/SMB dedup appliance i recommend LAN

Cheers,

Tristan.

[dellock6]

Uhm, not sure about the inline deduplication.
To date , the only appliance where both Veeam and the dedup vendor stated it can be left enabled is ExaGrid. Tristan, which dedup appliance are you using?

[Gostev]

So far, I have not seen a dedupe appliance where overall resulting data reduction ratio would suffer from keeping Veeam deduplication on. Remember that Veeam dedupe is done with large block size. At the same time, keeping Veeam dedupe on may speed up processing significantly.

Consider using Low compression as well, besides None. Low compression algorithm does not mess up advanced dedupe, but may speed up processing significantly.

[dellock6]

After several tests with ExaGrid, I found out the best combination is inline deduplication on and compression off.
But dedup appliances are really different one another, so everyone of them need best practices and vendor advice to be used at its best.
Do not know the official position by FalconStor.

[tsightler]

Inline deduplication on and compression off is pretty much the answer for every deduplication solution, however, some vendors focus heavily on wanting to show "maximum" dedupe ratio, so they suggest turning off Veeam dedupe. If your criteria for using a dedupe appliance is that you want the largest "dedupe ratio" possible, then of course you want to turn off Veeam dedupe. However, in most cases, the better goal is to focus on storing and transferring the least amount of data, as that will give the best performance, and in some cases, even be willing to give up a little dedupe/storage for performance.

To illustrate why some vendors like to recommend disabling Veeam deduplication consider this example. Let's say you are backing up a disk that has 40 identical 256K blocks, that's 10MB of identical data. Now lets assume that one of those blocks can be compressed and deduped 4:1 for a 75% savings. If you leave Veeam dedupe on Veeam will write only 1 of those 40 blocks, since the others are identical, that's a 40:1 savings in network traffic to the dedupe appliance, but the dedupe appliance will only report 4:1 dedupe. If we disable Veeam dedupe, Veeam will copy all 40 of the blocks to the dedupe appliance, thus copying 10MB instead of 256K, however, the dedupe appliance will deduplicate the 40 blocks, and report a dedupe ratio of 160:1. The amount of data on disk remained the same, but the amount transferred across the wire is 40x more. Once again, I recognize the contrived aspect of this example, but I'm only illustrating the point.

Using Veeam low compression can be a nice performance improvement, saving 10-20% in the total amount of data that must transfer across the wire, while increasing the size on disk typically by less the 5% do to slightly poorer dedupe. This 10-20% savings across the wire can really add up to some improved performance if you have many TB of full backups to run. Dedupe appliances can only ingest a fixed amount of TB's an hour and thus this performance improvement can translate into shorter backups, and faster restores due to less data having to be transferred across the wire during restores as well, but at the penalty of some additional loss of storage dedupe.

As with everything, it's all about striking the right balance for your environment.

[tfloor]

Uhm, not sure about the inline deduplication.
To date , the only appliance where both Veeam and the dedup vendor stated it can be left enabled is ExaGrid. Tristan, which dedup appliance are you using?

I WAS using a datadomain. DD610. Too slow. So I've placed it in /dev/null.

[stormlight]

I can't remember where I read this, or maybe watched it on a Veeam webinar:

Go to your Dedup appliance in Veeam under the backup repositories. Go to the advance tab where location is

Enable Align backup file data blocks
Enable Decompress backup data blocks before storing.

These two have helped me with my dedup box. DataDomain.

[Gostev]

Definitely avoid enabling backup file data blocks align for deduplication storage that uses variable block size dedupe (such as DataDomain). What saved you here is that this checkbox does not currently work until you add one missing registry key

[stormlight]

What you are saying is in conflict with this webinar or maybe the webinar was to generic and did not define the importance of storage that uses variable block size deDup. (I would hope that the presenter would outline that next time)

In addition the webinar makes no mention of the registry key.

http://go.veeam.com/webinar-12202011-ri ... am-ty.html

Is the decompress data blocks setting ok to check? Does this help with restores at all ?

Edit: I see the webinar was based on exagrid.

[Gostev]

Yes, not using compression is always a good idea with any deduplicating storage.

[dellock6]

As an addition for those using ExaGrid, you can ignore block align since ExaGrid uses variable size, compression is needed only if you have the ExaGrid off-site, so you're best compressing the backup in the local proxy and then de-compress at the target site before saving data to ExaGrid.
Uhm, I'm better write a blog post about these settings so I will not need to copy/paste them in so many threads

Luca.

[bengoa@tulane.edu]

[merged]

Is any one out there using a Deduplication Appliances with their VEEAM backups?

What appliance are you using? What kind of compression ratio are you seeing?

Are you backing up raw/ or are you using VEEAM deduping option as well?

We are looking at the DELL DR4000 backup appliance. Any insight would be appreciated.

Thanks,
Alex

[pcmmitham]

we're using steelheads for deduplication, nothing is better than them for that. but interacting with veeam....we're still working on it. its seemless for everyother app we use, but I think it might be whats causing veeam to fail miserably at replication. once we can replicate successfully I'll update this thread!!

[tsightler]

@pcmmitham - Are you referring to WAN deduplication? I think the OP was referring to deduplication storage appliances. If you are attempting to use Veeam replication with Steelheads than you'll want to disable multiple TCP connections as well as Veeam compression. Should work great. I have quite a few clients using the combination successfully.

[ibis69]

Hi, i'm using Datadomain DD670 appliance atm.

On veeam, storage options are configured like this :

- Compression Level : None
- LAN target
- Deduplication ON

We have 20 jobs, half of them are reverse incremental and others incremental with synthetics fulls
Retention set to 3 months
Compression ratio : 18X ( 163 To of data for 9 To used on datadomain)

[Gostev]

Nice! Can to share some more details please? We could use some real-world feedback since this is a common question.
- Are you getting any decent backup performance?
- How did you hook up DD to Veeam, via CIFS-based repository?
- Did you try running some VMs off those backups (any vPower feature)?
- Anything else worth sharing?

[NightBird]

Definitely avoid enabling backup file data blocks align for deduplication storage that uses variable block size dedupe (such as DataDomain). What saved you here is that this checkbox does not currently work until you add one missing registry key

So with a DataDomain appliance, should we use "Align backup file data blocks" as says "Veeam Backup & Replication Integration Guide, Version 6" or not ?

What are the Veeam Best Practices with DataDomain ?

[Gostev]

Dedupe-friendly compression, all other settings at default.