Deduplication appliance recommendation

Availability for the Always-On Enterprise

Re: Deduplication appliance recommendation

Veeam Logoby foggy » Mon Jan 25, 2016 9:25 am

I don't think you can get anything comparable to a dedupe appliance with a Windows box with deduplication enabled, however, per-VM backup chains and scale-out repository definitely allow you to get maximum of deduplicating storage in terms of backup jobs performance. As for Veeam inline deduplication, you can leave it enabled, but disable compression to achieve better deduplication rates.
foggy
Veeam Software
 
Posts: 15094
Liked: 1111 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Deduplication appliance recommendation

Veeam Logoby kte » Mon Jan 25, 2016 10:58 am

Storeonce VSA is also a software solution that is a VM and can be installed to 50TB of netto storage capacity. I just don't know if CAD duduplicates correctly
kte
Expert
 
Posts: 172
Liked: 7 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Deduplication appliance recommendation

Veeam Logoby Gostev » Mon Jan 25, 2016 1:21 pm

foggy wrote:I don't think you can get anything comparable to a dedupe appliance with a Windows box with deduplication enabled, however, per-VM backup chains and scale-out repository definitely allow you to get maximum of deduplicating storage in terms of backup jobs performance.

Actually, the dedupe ratio is very comparable (I would even say identical) as all the same algorithms are used anyway. And performance will always be great because Windows server dedupe is not inline - but rather post-process, so backup always land on raw storage at full storage speed (and chunked into the dedupe store later). Same principle as ExaGrid.

What is not comparable is functionality: there is source-side dedupe client (like DDBoost or Catalyst), no deduped volume replication, etc.

smd32 wrote:So per-VM backup file chains, scale-out repository, and leave dedupe to Windows Server 2012 R2 or 2016 instead of Veeam -- gets you most of the functionality of a dedupe appliance? Or go with the first two and stick with Veeam dedupe but just spend the extra cash for more disk?

Windows Server 2012 R2 dedupe does not scale well, and is typically not recommended by our users who actually tried to use one on more than 5TB of VMs. I've seen occasional reports about 10TB and even more, but this required lots of tweaking.

Windows Server 2016 dedupe is yet to be released and seen in action - until then, no recommendations can be made.
Gostev
Veeam Software
 
Posts: 21517
Liked: 2383 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Deduplication appliance recommendation

Veeam Logoby mkaec » Mon Jan 25, 2016 2:24 pm

The missing piece on Microsoft's side is post-deduplication replication. If one were to try to stay within the Windows tools, DFS replication would be used to replicate to an off-site location. Unfortunately, DFS-R currently rehydrates before replicating. I think even using Veeam to do the replication would still require rehydration from a Windows dedup store.

But, I think Windows dedup would work very well in cases when you don't need to replicate. (Such as if you are backing up files that have been received from Remote locations via DFS-R.)
mkaec
Expert
 
Posts: 185
Liked: 48 times
Joined: Thu Jul 16, 2015 1:31 pm
Full Name: Marc K

Re: Deduplication appliance recommendation

Veeam Logoby Gostev » Tue Jan 26, 2016 1:20 am

Correct. I would use it as a secondary backup repository to target Backup Copy jobs to.
Gostev
Veeam Software
 
Posts: 21517
Liked: 2383 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Deduplication appliance recommendation

Veeam Logoby chimera » Tue Jan 26, 2016 8:49 am

We have been looking at EMC Data Domain DD2500 with DDBoost as a possible solution, however this is quite expensive

EMC DD with DD boost gets my vote (yes they are expensive, but you get what you pay for)
They are brilliant boxes. Set and forget. We get insanely fast backups to it from Veeam.
chimera
Enthusiast
 
Posts: 57
Liked: 3 times
Joined: Thu Apr 09, 2009 1:00 am
Full Name: J I

Re: Deduplication appliance recommendation

Veeam Logoby stephane.duprat » Mon Feb 01, 2016 9:05 am

Hi,
I'm using 2 DD 620 and they get my vote too.
No problem with replication. Deduplication very good. EMC support OK.
I'm interrogating me for change to DD2500
But only for backup. Restore take long long time.

Chimera, I would know if with DD 2500, restores are better (Granular files or email or SharePoint objects).

Thanks
stephane.duprat
Lurker
 
Posts: 1
Liked: never
Joined: Thu Jul 24, 2014 7:30 am
Full Name: Stéphane DUPRAT

Re: Deduplication appliance recommendation

Veeam Logoby lightsout » Mon Feb 01, 2016 2:37 pm

I've got a new Quantum DXi 4700, and that performs really pretty well. I am only doing backup copies to it, but it ingests data as fast as I can throw at it!

Doesn't have the cool integration that the Data Domain does, but 10Gb CIFS connectivity works well enough for my needs. I actually really like the look of Exagrid's appliances too.
lightsout
Expert
 
Posts: 186
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Deduplication appliance recommendation

Veeam Logoby rreed » Mon Feb 01, 2016 3:30 pm

Having nothing but trouble from our DD's, though it's been since found the problem may not have been quite as much w/ the devices directly, but then again we do still see occasional issues w/ ours that are resolved w/ nothing more than a reboot - we thought we'd give the Dell DR's (DR4100 in our case) a try. At a fraction of the up-front cost, a fraction of the yearly support renewals, native 10Gb, and better performance once the data gets onto the device, we will not be renewing our sky high DD support and simply letting them die on the vine as unsupported misc. archival storage. Granted, Dell has recently bought EMC so not sure how much of a moot point it would be anyway. :wink: Our Dells came in at about 1/3 of the cost of our old EMC's and been much more stable and much faster.

Now, as far as dedupe appliances, based on my experience I wouldn't recommend on buying into the fad for any manufacturer. Before walking in the door here I had about 4-5 years' experience w/ Veeam and the old classic staging to plain old Windows storage first, then pull that off to tape for archival. It just worked, rock solid, for years. We'd keep about two weeks on disk since that's into what most of our restore requests would fall, plus of course last night's disaster recovery. We had a very open expectation w/ users if we had to go to tape, it would take longer, especially if it was off-site tape. No problem, everybody was happy.

My current company had long before since into the dedupe sensation w/ two pairs of DD620's and a pair of DD2200's (two data centers w/ matching sets, cross-replicated at the DD level) before I walked in the door. As mentioned lots of writing problems that have since turned out to be a VMware NIC driver problem, but still occasionally like to fail authentication, which fails the backups for that night - or weekend, albiet a simple reboot the next morning fixes it until the next time. We have had to remove/re-add to AD authentication when they get *really* broken a few times but it's a quick fix. Still, "reliability." Now, speed. Dedupe devices will ingest really well, and that's what the marketing pushes. Try a restore and you'll be waiting around for a very long time. Setting your jobs to largest block size (16+TB) will help, but it's nowhere near pulling directly from disk. Start a VM from backup? No, though Veeam indicates there's a better chance of it happening w/ V9. Everyone here has been appalled w/ restore speeds, and we once had an emergency restore of about 2TB of our main file server. Took a couple of days. File level restore? What used to take around 10-15 min. back in the day of just plain old disk storage now takes no less than half a work day from dedupe. Maybe more if it bombs out half way through, the user says no that's not the file, etc. I've since been able to tune and tweak to get it better, but it's still dismal. And here we're writing straight to dedupe which is not best practice. I've tried pushing getting some staging disk space to help mitigate emergency restores but no, there's still the expectation that dedupe devices are perfect. In the end, they are effectively a one-way street of data storage. They'll store great, just don't expect to get much back out of it, at least not any time soon.

I get the dedupe/compression aspect - about 86%-88% real-world (despite what the sales manager tells you). That's fantastic, we're storing hundreds of TB's in just around 30-40TB of raw dedupe storage. I completely get the attraction. Veeam's dedupe and compression may not quite hit those marks but v9 has really slugged the crap out of it, I've seen our v9 final vbk's and vib's shrink to 1/3-1/4 of their previous v8 size in some cases due to dirty blocks being written to storage in previous versions. However, what is the cost of a large JBOD vs. even a Dell DR, or especially EMC DD? I bet a big tray crammed full of disks and v9's much better storage capabilities would win way out over a dedupe box - and you can actually restore files/VM's, run a VM from backup(!!!), more easily manage, etc. I would recommend staging JBOD w/ v9, and final archival to tape. Walk away from the cool aid of dedupe devices. They're marketed quite well but just don't deliver back to you when you need it back out of them. You asked for recommendation, these are my opinions based on experience.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)
rreed
Expert
 
Posts: 354
Liked: 72 times
Joined: Tue Jun 30, 2015 6:06 pm

Re: Deduplication appliance recommendation

Veeam Logoby ChuckS42 » Mon Feb 01, 2016 7:03 pm

All I will say is to NOT get a Dell DR4100 or DR6000, until and unless Veeam officially supports them. Very painful to use with Veeam right now.
ChuckS42
Enthusiast
 
Posts: 66
Liked: 5 times
Joined: Wed Apr 24, 2013 8:53 pm
Full Name: Chuck Stevens

Re: Deduplication appliance recommendation

Veeam Logoby rreed » Mon Feb 01, 2016 7:26 pm

Dang, hate to hear that Chuck. We've had opposite experience here w/ the Dells vs. EMC. Our DR4100's have been solid and reliable vs. the DD's, but admittedly our old DD's are hanging off Cisco 3750 switches which do not have adequate buffers to handle the throughput (high output drops at the DD switch ports). And we're piping (2x) 10G LAG from our SAN/VMware infrastructure to the 3750's, from there just 1G ports to the DD's so probably not fair. I'm in process of moving the DD's 1G over to our 10G core via mini GBIC's which has solved the output buffer problem but we still get the occasional "I'm not going to authenticate anyone until someone reboots me" w/ the DD's. Dells are hanging off the same 10Gbps core and seem to keep up w/ ingesting our backups just fine.

Anyways, if you're going to buy into deduplicating devices, please make sure your infrastructure can handle the throughput required for backups to run. Or any storage device for that matter. We had a lot of long-standing backup issues that had us pointing our fingers at the storage device until we figured out we were using desktop wiring closet switches in our data center and some old pNIC drivers on the VMware side were plaguing us w/ endless disconnects.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)
rreed
Expert
 
Posts: 354
Liked: 72 times
Joined: Tue Jun 30, 2015 6:06 pm

Re: Deduplication appliance recommendation

Veeam Logoby pirx » Mon Feb 01, 2016 10:25 pm

Any opinions in StoreOnce, especially the 6500? We already use some smaller modelles at remote sides withot Veeam, but we have 0 experience with other vendors.
pirx
Enthusiast
 
Posts: 69
Liked: 7 times
Joined: Sun Dec 20, 2015 6:24 pm

Re: Deduplication appliance recommendation

Veeam Logoby ChrisSnell » Tue Feb 02, 2016 1:17 pm

The most important consideration is not backup performance, but restore performance. Because ExaGrid is the only dedupe device to use a landing zone, it can recover from this 'normal disk' area without having to rehydrate deduped VMs. This gives a much quicker recovery, and so reduced downtime. The integration of a Veeam data mover in to the appliance also massively helps in terms of backup speed (1.6x faster than to CIFS) and also synthetic full creation (6x faster than using a proxy).

The landing zone, and then post-backup dedupe process in to a retention zone, also provides it with some of the quickest backup performance - this obviously depends on appliance.

Having been an SE at Veeam for 4 years, and now at ExaGrid - hopefully folks will be able to take my word. Happy to discuss further with anyone.
ChrisSnell
Technology Partner
 
Posts: 117
Liked: 14 times
Joined: Mon Feb 28, 2011 5:20 pm
Full Name: Chris Snell

[MERGED]Re: Recommendations for backup storage, backup targe

Veeam Logoby sdelacruz » Thu Sep 21, 2017 5:02 am

Need to decide for a new data repository. My goals is to have the fast recoveries.



Dell Windows Server 2016 refs with 16 4TBs NLSata vs Raid 10 vs. EMC Data domain 2200.

Do deduplication appliances perform well on recovery speed?

Sam
sdelacruz
Enthusiast
 
Posts: 46
Liked: never
Joined: Tue Feb 01, 2011 8:09 pm
Full Name: Sam De La Cruz

Re: Recommendations for backup storage, backup target

Veeam Logoby DGrinev » Thu Sep 21, 2017 11:07 am

Hi Sam,

In general deduplication appliance has a negative impact on the recovery performance, since deduplicated data blocks should be rehydrated before the restore (unless it has a so called landing zone that allows increase performance of restore process for the recent restore points).
Please review this thread for additional information. Thanks!
DGrinev
Veeam Software
 
Posts: 538
Liked: 63 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: khang1231 and 46 guests