Unitrends/PHD Virtual calling out Veeam's dedupe

[Daveyd]

http://www.phdvirtual.com/Q22014-Veeam-Dedupe

Pretty ballsy if you ask me.

On the other hand, is Veeam going to ever implement a "global" dedupe feature or is it simply not needed in the current Veeam architecture?

[Gostev]

100% marketing speech that most of our "global" dedupe competitors absolutely love to use against us.

In reality, there is no difference between their "global" and our "per-job" dedupe. Let me do some exorcism now, because there have been too many marketing demons around this particular topic lately, and most of our competitors love to employ those demons in their messaging against us.

There are three types of dedupe that I commonly refer to as:
Type 1. "Per-job": such as in Veeam
Type 2. "Global" (marketing): such as in EMC Avamar, VDP(A), PHDVirtual
Type 3. "Global" (true): such as provided by deduplicating storage devices

Further in this post, when referring to "per-job" and "global" dedupe, I will be talking about Types 1 and 2 from this list.

Fact 1. Obviously, there is absolutely nothing preventing you from creating a single job with all your VMs in Veeam, effectively making "per-job" dedupe "global". I could really stop the argument at this point, if exorcism was not required.

Fact 2. In any case, both "per-job" and "global" dedupe approaches will be limited by the same thing: backup repository size, that in turn depends on a storage device capacity. No backup vendor out there provides dedupe that is truly global, its only global within the given dedupe repository. Now, all economical backup storage options will be of relatively small capacity (because if you have big bucks to spend, you cannot beat cost per TB of deduplicating storage, and then this discussion becomes a moot point). This means that in real world, every solution will be forced to use multiple separate targets, effectively working in "per-job" deduplication mode (one job per target). And just like in case with Veeam, "global" dedupe solutions will too have identical data sitting in those separate targets. As far as small shops that only have a few dozens VMs and thus can get away with a single target, see Fact 1.

Fact 3. What "global" dedupe vendors are NOT telling you, is that in addition to being limited by the backup repository size, there is also maximum dedupe store size limitations in the actual software. Since I do not follow lesser ankle biters like PHD closely, I don't know specifics there. What I do know, is that EMC engine-based products (Avamar, VDP and VDPA) have from 2TB to 8TB limit per their "global" dedupe store.

Now may I ask, what kind of global dedupe is that even with 8TB limit. They cannot be serious? Because this means that in any decent size environment, even if you DO have a single huge repository available, the user will be forced to create dozens of separate dedupe stores on it. Such dedupe cannot be considered global no matter how you define the word "global", and overall disk space usage of this approach will be exactly the same as with dozens of Veeam jobs pointing to these same "sub-repositories".

But wait! There is no need for dozens of Veeam jobs, because Veeam does NOT have similar 8TB limitation! We can handle hundreds of TB in a single job, if you like! This means that in reality, Veeam's "per-job" dedupe is much more "global" than one of those vendors who claim to have global dedupe "unlike Veeam". What a sudden turnaround!

To summarize:
1. Through "per-job" dedupe, Veeam already provides "global" dedupe of Type 2 that is at the very least not worse than one of vendors claiming to provide "global" dedupe. And in many cases, our dedupe can be made much more "global" so to speak, if so desired.
2. We believe that the right way to do true global dedupe is Type 3 (by the storage device). Primarily for reliability, performance and economical aspects. And tighter integration with such storage devices is one of the major R&D investment areas for us in the upcoming v8.

Note that I did not even get to benefits of performing deduplicated backup into a regular file (self-contained, portable) vs. into software dedupe store (immovable, pinned down to the storage) like all those "global" dedupe vendors happen to do, and how our approach helps to address various real-life DR needs spanning anyone from SMB to Enterprises... but that's totally different topic, and this post is already getting way too long.

Hope this helps!

[Daveyd]

Awesome post! That's why I love Veeam!

[michaelryancook]

Excellent and informative post Anton.

I do see one scenario where this explanation does not fit and where I see a need for dedupe over multiple jobs. That would be for those of us still hampered by long term archival requirements that can only afford tape as we do not have disk space to retain years of data. As tape jobs require the entire .vbk to be restored for a single file or VM, we must break our backup jobs out into smaller jobs in order to reduce required disk space during a restore as well as to meet RTOs. It would be very nice to have dedupe over multiple jobs for this as it would help keep disk requirements to a minimum while still maintaining all of our tape restore requirements.

Now I guess there are two solutions to this unique scenario:

1) Dedupe over multiple jobs. Not necessarily global dedupe but at least dedupe a group of jobs on a single repository.
2) File-level or VM-level granular restore from tape. Which I have already put my comments in on this thread.

Thanks, Michael

[Gostev]

Hi, Michael. We do plan to provide restore directly from tapes in the future versions. Thank you for your feedback!