Comprehensive data protection for all workloads
Post Reply
scottf99
Enthusiast
Posts: 55
Liked: 4 times
Joined: Jul 29, 2013 2:13 am
Full Name: Scott
Contact:

Deduplication Single Server

Post by scottf99 »

Hi. I have a Backup Exec background so please excuse if I misunderstand Veeams Deduplication.

I have a client with about 30 Vmware Guests and 2 Hosts. Using Veeam 6.5 I backup 19 of these guests in one job and it all works fine.
The 20th Server has about 4TB of flat file data. I back this up with a single job.
With Backup Exec if I backup this Server to a dedup location it will take up 4TB, same as with VeeamB&R.
If the next day I do another Full backup with Backup Exec it will not take up much more space on the dedup location (as it actually dedupes the flat file data). If i do the same with Veeam it create a whole new 4TB file. This is causing problems as I only have 10TB of space for the Veeam backups and client wants to keep 6 months of backups. Even if I only do a full backup every 2 months i am still screwed. I have looked into Reverse and Syntethetic but it creates a lot of CPU usage and the NAS repository is overwhelmed with IO.

Am I doing something wrong or do I just need to look at how Veeam deduplicates a single Server (I understand and see Veeam deduplication working with 2 or more similar Servers).

Cheers
Scott
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deduplication Single Server

Post by veremin »

If the next day I do another Full backup with Backup Exec it will not take up much more space on the dedup location
I’m wondering what dedup location you’re talking about. Is it an appliance that is capable of deduplication or it’s more about Backup Exec deduplication option? Thanks.
scottf99
Enthusiast
Posts: 55
Liked: 4 times
Joined: Jul 29, 2013 2:13 am
Full Name: Scott
Contact:

Re: Deduplication Single Server

Post by scottf99 »

Thanks for your reply.
The reference to BE was just to show what I am used to.
Veeam is backing up to a NAS through iscsi. The LUN appears as a 15TB NTFS formatted partition on the Veeam Server/Proxy.
Veeam Server/Proxy is Server 2008R2 as are the guests being backed up.

I guess I could simplify my question by asking "where is Veeam deduplication?" It is advertised as being a major feature but if I do 2x 4TB Full backups it takes up 8TB backup space. That is not deduplication???

Cheers
Scott
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deduplication Single Server

Post by veremin »

Hi, Scott. I believe there must have been a slight misunderstanding on how VB&R deduplication works. In fact, deduplication doesn’t work between the VBK and VIB files; instead, it works only within each VBK and VIB.

Other things related to deduplication are pretty good covered in sticky FAQ; so, it might be worth taking a look.

Thanks.
scottf99
Enthusiast
Posts: 55
Liked: 4 times
Joined: Jul 29, 2013 2:13 am
Full Name: Scott
Contact:

Re: Deduplication Single Server

Post by scottf99 »

Thanks again. I am sure it is my misunderstanding, I just wanted confirmation. I have read all the guides and FAQs.

My understanding of deduplication is when a block (byte, bit, whatever) is written to a backup then it is not written again, just a link or pointer to it is used. I hope this is correct.

When Veeam does a full backup and then another identical full backup does it not do this? Does it just create another backup file?

If I now understand correctly then my question is ... why? Why not just write to the same file and actually deduplicate the data. What is happening is a duplication.

Apologies if I am wrong.

With all the terrible problems with BE lately Veeam seems to be the way to turn (and I have purchased it for one client so far) but this is a major failing and I was genuinely hoping I was just doing something wrong.

Cheers
Scott
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deduplication Single Server

Post by veremin »

As mentioned, with forward incremental mode deduplication exists only on one restore point level. So, there is no deduplication between previous restore points and newly-created one. In other words, there will be a deduplication between VMs within one restore point, meanwhile, there will be none between restore points themselves.

Your understanding (though, being not 100% true) is related mostly to reversed incremental mode, where the recent restore point is always a full backup that gets updated after every backup cycle. With reversed incremental changes (new blocks) are injected into the .vbk file to rebuild it to the most recent state of a VM and the blocks that have been already present (latest restore point - .vbk file) are skipped.

However, even with reversed incremental if you decided to perform a new active full backup, there would be deduplication only across VMs that are being backed up, not across existing restore points.

Thanks.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplication Single Server

Post by Gostev »

Use reversed incremental backup mode (recommended backup mode for disk-based targets). This mode is designed to maintain a single full backup on disk no matter of retention policy. This is just a different approach to the same problem (avoiding storing multiple full backups on disk). Symantec uses target-side dedupe to only store full backup data once, while we use reversed incremental backup mode instead. You will find that our approach is much more efficient, because it does not require periodic fulls, which can take a very long time. It is not uncommon that full backup takes a few days in some environments, so periodic fulls are simply impossible there.
scottf99
Enthusiast
Posts: 55
Liked: 4 times
Joined: Jul 29, 2013 2:13 am
Full Name: Scott
Contact:

Re: Deduplication Single Server

Post by scottf99 »

Thank you both for your explanations and for the solution. I will give reverse incrementals a go again.

Cheers
Scott
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deduplication Single Server

Post by veremin »

The best method, indeed, in terms of storage saving. Should any additional help be needed, don’t hesitate to let us know. Thanks.
benthomas
Veeam Vanguard
Posts: 39
Liked: 11 times
Joined: Apr 22, 2013 2:29 am
Full Name: Ben Thomas
Location: New Zealand
Contact:

Re: Deduplication Single Server

Post by benthomas »

Another option would be to upgrade your backup server to Windows 2012 and install the data deduplication role on it. Then you could enable Deduplication on the iSCSI drive and have it run a deduplication run every night and it wouldn't matter how many full backups you did with Veeam, because Windows would detect the duplicate blocks and reduce the required space.
Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deduplication Single Server

Post by veremin »

Another option would be to upgrade your backup server to Windows 2012 and install the data deduplication role on it.
Though, in this case the recommended backup method would be forward incremental, instead of, reversed one, since the forward incremental is supposed to guarantee higher deduplication rates. Moreover, if you’re after using restore and/or surebackup options, be aware to set OS to dedupe only files that are older than a specified interval.

As to other users’ experience regarding MS 2012 Deduplication, it’s pretty good covered here.

Thanks.
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 287 guests