Comprehensive data protection for all workloads
Post Reply
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Deduplicating Backup Storage Suggestions

Post by mongie »

Hello,

We're currently in need of some extra storage for our Veeam backups, and I'm thinking along the lines of a de-dupe appliance. We're currently storing ~ 60TB of backups on 2x Dell MD1200 direct attached shelves with 3TB disks in RAID 50. I find that the MD's are (I assume this is the problem) too slow to do surebackup restores on my high IO servers (specifically Exchange 2003).

We' ve identified that we're going to need to store at least 120TB of backups on the new appliance. Ideally, we'll be
So far I've spoken to Dell - we normally use their gear - about the DR4000 appliance. The model in question will have 18tb of disk and is supposed to store 270TB. We're also looking at a datadomain (or potentially anything from IBM, HP or NetApp). I've only got pricing on the Dell unit so far, and it is probably within my budget (around 35K AUD).

Does anyone have any advice on what to buy or what not to buy? I've seen threads on here criticising datadomain for slow ingest rates... and I think I may have seen the same mentioned about HP. From my brief conversation with the vendor, the DD model we'd be looking at is the DD640. Is there any support for DD Boost within Veeam? HP (Catalyst) software? Is that on the roadmap? I know that Exagrid are support Veeam, but they dont appear to be expandable, and thats one of the things I have been looking at as a positive. My vendors have also not really heard of them. I know they have postprocess dedupe which is different to most of the other vendors... What about FalconStor, Quantum?

Compared to plain DAS storage, are there any major differences with using Veeam with a dedupe appliance as the storage? Is there anything (apart from space and ingest rate) that I should be considering? Is it possible to run surebackup restores from the appliances?

Any advice you can provide would be appreciated.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Gostev »

Hi,

Massive Exchange 2003 servers is just naturally bad workload for vPower (since Exchange 2003, Microsoft has reduced Exchange I/O requirements a few times both in Exchange 2007, and especially in Exchange 2010 - one of the major reasons why we upgraded, btw).

Your current storage is certainly much faster than any inline deduplicating storage mentioned above (pretty much any raw storage is much faster than deduplicating storage). Typically, you are not able to run SureBackup from any inline deduplicating storage. So one thing you should not hope is that this new storage will improve vPower performance. ExaGrid is different though, as it does post-process dedupe and features raw disk landing zone.

I believe ExaGrid's main differentiator is expandability (thus "grid" in their name), but you'd better check with them directly.

At this time, we do not plan specific integrations with any backup storage - we like to remain storage-agnostic, and happy with the current performance of writing to those boxes anyway. While integration will not help with vPower performance anyway, because it does not accelerate random I/O.

Regarding best settings for writing to dedupe devices, they depend on device - most vendors provide integration guides, and their recommendations is usually different from ours because of different goals ;) anyway, there are plenty of existing discussions regarding this here on these forums, so please just use search to find those.

Thanks!
deduplicat3d
Expert
Posts: 114
Liked: 12 times
Joined: Nov 04, 2011 8:21 pm
Full Name: Corey
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by deduplicat3d » 1 person likes this post

I would recommend not getting a dedupe device. I use dedupe for the sole reason that I am required to send backups to tape which works best with incremental (weekly fulls). If I were you I would get some really fast backup storage and just do reverse incrementals.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Gostev »

Indeed, deduplicating devices are best used for the long term data archival purposes (as a tape substitute).
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by mongie »

So you're suggesting that I would continue backing up to raw disk, and then archive older backups somehow on the dedupe appliance?

Is there anything the roadmap around making it easier to transfer backups around without breaking the database? At the moment that is looming as a complication to using multiple types of disk... I'm using forward incremental with active fulls at the moment and its working fine... I dont really want to use reverse inc if I can get away with it because the extra IO makes the backups SLOOWWW.

As far as buying really fast storage - well I'd love to, but try getting the $100k or whatever it would cost for 190TB of disk signed off :D

Looks like I'll just do a POC on a datadomain or something and see how I go.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Gostev »

mongie wrote:Is there anything the roadmap around making it easier to transfer backups around without breaking the database?
Please clarify a little bit what you mean here.
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by mongie »

At the moment, you can move a whole backup job to another location... but it would be good if somehow I could move parts of a job...

Archive locations would be ideal... e.g. keep x restore points on location 1 and then y restore points on location 2.
kortasma
Lurker
Posts: 1
Liked: never
Joined: Aug 07, 2012 1:39 pm
Full Name: Matthew Kortas

Re: Deduplicating Backup Storage Suggestions

Post by kortasma »

Another idea is to consider an appliance that enables deduplication at the primary location of the VMDKs. A NetApp FAS unit can be deployed with low cost storage. The deduplication space savings occurs at the source, so you consume less capacity when storing the VMDKs. And, you can also enable disk based snapshot to protect against logical failures. You can then look at a less expensive secondary disk solution to simply write full backups to.

My proposed solution works in reverse too, you can configure at NetApp FAS to receive backups via NFS, CIFS or SAN protocols and do post-process dedupe and/or inline compression. However, the NetApp FAS device is better suited to serving primary data for VMs versus being relegated to a secondary disk solution. Good luck!
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by mongie »

I'm going to be running a POC with a data domain appliance. Their "key" to throughput is the number of streams being written to disk at once (up to 90?) so, is there any way to know or adjust the number of streams being written to disk by veeam?

Its a shame that you're not looking at implementing DD Boost, it looks like it provides a good benefit - BackupExec and vRanger both support it.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Gostev »

1 stream per job.

DD Boost will certainly provide a good benefit for any solution that does not feature inline source-side dedupe and compression, like the above-mentioned products (and unlike Veeam).
Saintly
Novice
Posts: 5
Liked: 1 time
Joined: Oct 16, 2012 9:33 pm
Full Name: Ian McGuinness
Contact:

[MERGED] Backup targets

Post by Saintly »

Hi,
I am currently making selections to replace my backup systems including hardware and software.
In this thread I was choosing between AppAssure and Veeam and have decided on Veeam.

Now i am considering the backup target to save the backups too.
Dell is pushing their DR4000 Disk Backup Appliance. It has features like deduplication, compression and replication built in.
It has 18Tb of space within it but the Dell tech rep claims that with the deduplication and compression that it's actualy equivilant to about 270Tb of storage. I personaly don't understand how it can deduplicate backup files as each file would be unique wouldn't it?
I would still be asking Veeam to deduplicate and compress so the amount of data being pushed accross the network is smaller so im not sure if the DR4000 would find more that it could reduce.
They also say that a benifit is that having the device handle the replication to another site (a second DR4000) frees up the backup software and reduces our backup window.

Does anyone have experience with this device or does anyone think i should just get large disk arrays with heaps of disk space and rely on Veeam to handle the deduplication, compression and replication?
Any other options that i should consider?

A bit about my current environment:
I have 3, 2 socket servers running VMware Essentials Plus ESXi (i.e. 6 sockets)
These connect to a SAN (Dell MD3200) to house the VMs
Current backups is with BackupExec to a tape autoloader. Very slow and clunky. Not really looking after the Linux servers.
Running 8 Windows 2008r2 VMs and about a dozen Linux VMs.


Thanks
Ian
dellock6
VeeaMVP
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by dellock6 »

Hi, I've never used the Dell unit, but I have a fair experience with deduplication appliances, mostly DataDomain and ExaGrid.
First, do not trust their numbers: usually are there as "best case". If you fill them with txt files deduplication is huge and can probably reach those numbers, but what if you save zip files or jpeg images, already compressed? Your best choice is to ask them for real numbers about Veeam backup files.
Also, remember Veeam does not do file backups, it saves whole VMs, so deduplication must occur at byte level. So it does not matter if there are unique files or duplicated ones, dedup happens at the block level.

About replication, they are correct: since Veeam can save only to a single destination for any given job, you can run Veeam only once and save to the first appliance, and then let the appliances replicate.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Vitaliy S. »

Ian, I would also recommend taking a look at Windows Server 2012 as a storage device for your backup files, with deduplication feature enabled you should get pretty decent global deduplication results.

For further reading, please check out our blog post: http://www.veeam.com/blog/how-to-get-un ... ation.html
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Gostev »

Windows Server 2012 dedupe might not suit well depending on amount of data that needs to be backed up, as it has pretty limited post-process dedupe performance. So, I would not compare it directly to hardware storage appliances.
jpeake
Enthusiast
Posts: 88
Liked: 25 times
Joined: Sep 25, 2012 7:57 pm
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by jpeake »

Saintly

I just went down this path with the DR4000 also. The claims sound nice, but in the end Dell wasn't able to prove it with real-world data. I asked Dell for a demo unit, they don't offer one. They did send me a spreadsheet that the Dell storage engineers use to size systems. The macro's are locked so i can't see the code (was hoping to find the formulas). But entering in our estimated workload data - it came back with an expected dedupe ratio of 5:1. A far cry from the 15:1 they claim in the marketing materials.

So I ended up ordering a Dell MD3600i, using 3TB disks and 10Gbe. It's not been delivered yet, but will let you know how it goes when I get it. I was looking at the 5.4TB/70TB DR4000. The MD3600i was about $4000 cheaper than the DR. Will use it combined with Veeam's dedupe, hosted on a Win 2012 box using Windows dedupe. Veeam seems to be pushing this combo (not the storage hardware, but the combo of Veeam and Win2012). Check out the "whiteboard Friday" from last week. They detail the setup pretty well.

The dedupe ratios are pretty dang nice with this combo, and seems to have less headaches than you MIGHT have with dedupe appliances. And it's faster and cheaper.
chimera
Enthusiast
Posts: 57
Liked: 3 times
Joined: Apr 09, 2009 1:00 am
Full Name: J I
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by chimera »

Dell is pushing their DR4000 Disk Backup Appliance. It has features like deduplication, compression and replication built in.
It has 18Tb of space within it but the Dell tech rep claims that with the deduplication and compression that it's actualy equivilant to about 270Tb of storage. I personaly don't understand how it can deduplicate backup files as each file would be unique wouldn't it?
Not done at file level. It only keeps unique blocks of data with an index/pointers to each 'unique' block thus saving on storage space.
I would still be asking Veeam to deduplicate and compress so the amount of data being pushed accross the network is smaller so im not sure if the DR4000 would find more that it could reduce.
On the contrary. Dedup appliances are purpose built for just that, dedup'ing and compressing data. Even with Veeam dedup/compression on, I would guarantee you'll get further deduplication/compression on a dedup device. The biggest benefit I personally see with the DR4000 (or in fact, any dedup device that supports it - I've only had experience with DD) is the replication feature by automatically replicating dedup'd/compress blocks of data across the LAN/WAN to a similar device (preferably) in an offsite location. To me thats a big selling point, especially when I've got 30TB of data (if running a full) to backup. Fire in the server room? Flood? Earthquake? Meteorite? :D Having offsite backups largely "automated" (via the replication) equals peace of mind.

Of course, if you're running EqualLogic currently and aren't in a hurry, then wait til Dell port the Ocarina dedup code into the EqualLogic firmware, add a member (tray) of SATA disk, utilise that as a backup target, then buy another member house it offsite and utilise EqualLogic SAN replication to transfer your backups offsite.
emachabert
Veeam Vanguard
Posts: 388
Liked: 168 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by emachabert »

Hi,

We are using two HP D2D 4112 (9TB) replicating between two sites over a 40mbit/s optic fiber line (150 km), those boxes are only used as Veeam target:
Here is our global dedup ratio:
Disk Space Used 3.5 TB of 8.9 TB (39.6%)
Deduplication Ratio (7.0 : 1)
User Data Stored 24.7 TB

Here are the stats for each CIFS share:
VEEAM-LINUX 29 restore points (real data: 2.7 TB, size on disk: 288.7 GB, ratio 9.5:1)
VEEAM-WINDOWS 37 restore points (real data: 7.1 TB , size on disk: 1.0 TB, ratio 7.0:1)
VEEAM-SHAREPOINT 43 restore points (real data: 12.2 TB , size on disk: 1.2 TB, ratio 10.2:1 )
VEEAM-EXCHANGE 18 restore points (real data: 2.7 TB , size on disk: 753.9 GB , ratio 3.6:1 )

Write speed is between 60 and 80 MB/s.

The only problem we are facing at the moment (we have an ongoing support ticket with both Veeam and HP) is read speed.
We can't really use vPower NFS based jobs as random reads are too bad (5 hours to restore an item from our sharepoint app).
Veeamizing your IT since 2009/ Veeam Vanguard 2015 - 2023
dellock6
VeeaMVP
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by dellock6 »

I suspect HP appliance is doing inline dedup, right? Read speed is always a problem with inline, since it needs to rehydrate data before exporting it. And vPower + Instant VM Recovery is almost the worst foe of this configuration :)
BTW, the debate about inline vs post-processing deduplication is almost never ending, like discussions about religion...

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Rumple
Service Provider
Posts: 81
Liked: 14 times
Joined: Mar 10, 2010 7:50 pm
Full Name: Mark Hodges
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by Rumple »

jpeake wrote:Saintly

I just went down this path with the DR4000 also. The claims sound nice, but in the end Dell wasn't able to prove it with real-world data. I asked Dell for a demo unit, they don't offer one. They did send me a spreadsheet that the Dell storage engineers use to size systems. The macro's are locked so i can't see the code (was hoping to find the formulas). But entering in our estimated workload data - it came back with an expected dedupe ratio of 5:1. A far cry from the 15:1 they claim in the marketing materials.

So I ended up ordering a Dell MD3600i, using 3TB disks and 10Gbe. It's not been delivered yet, but will let you know how it goes when I get it. I was looking at the 5.4TB/70TB DR4000. The MD3600i was about $4000 cheaper than the DR. Will use it combined with Veeam's dedupe, hosted on a Win 2012 box using Windows dedupe. Veeam seems to be pushing this combo (not the storage hardware, but the combo of Veeam and Win2012). Check out the "whiteboard Friday" from last week. They detail the setup pretty well.

The dedupe ratios are pretty dang nice with this combo, and seems to have less headaches than you MIGHT have with dedupe appliances. And it's faster and cheaper.
Did you get this up and running yet? Curious about results and amount of data you can retain.
cramry
Novice
Posts: 6
Liked: never
Joined: Feb 04, 2014 5:31 pm
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by cramry »

"We are using two HP D2D 4112 (9TB) replicating between two sites over a 40mbit/s optic fiber line (150 km), those boxes are only used as Veeam target:
Here is our global dedup ratio:
Disk Space Used 3.5 TB of 8.9 TB (39.6%)
Deduplication Ratio (7.0 : 1)
User Data Stored 24.7 TB

Here are the stats for each CIFS share:
VEEAM-LINUX 29 restore points (real data: 2.7 TB, size on disk: 288.7 GB, ratio 9.5:1)
VEEAM-WINDOWS 37 restore points (real data: 7.1 TB , size on disk: 1.0 TB, ratio 7.0:1)
VEEAM-SHAREPOINT 43 restore points (real data: 12.2 TB , size on disk: 1.2 TB, ratio 10.2:1 )
VEEAM-EXCHANGE 18 restore points (real data: 2.7 TB , size on disk: 753.9 GB , ratio 3.6:1 )"

Could you elaborate, if possible, on your backup to the HP D2D? I only ask due to I'm currently using 2x D2D4106 (8TB) for offsite backups and the dedup I'm getting is absolutely terrible! I've tried multiple scenarios with no improvement. Current Scenario is that I have about 2+ TBs disk space used, with 6TBs user data stored, and total storage is showing at 6TBs. I contacted HP who stated that the difference was due to the complex calcualations and the "metadata".
emachabert
Veeam Vanguard
Posts: 388
Liked: 168 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Deduplicating Backup Storage Suggestions

Post by emachabert » 1 person likes this post

Here is the link to the white paper written on that setup :
http://www.veeam.com/wp-veeam-hp-machab ... goals.html
Veeamizing your IT since 2009/ Veeam Vanguard 2015 - 2023
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 113 guests