Veeam & Deduplication Appliances

[Jpurdy79]

Long story short, I'm looking to move towards Veeam for our backups. However, at the same time, I'd like to leverage a deduplication appliance. I'm currently looking at the following appliances:

Quantum DXi
EMC Data Domain
FalconStor FDS
ExaGrid EX

I was hoping to hear anyones opion, experiences, strifes with any of these appliances working in conjunction with Veeam Backup and Replication.

[Gostev]

I would say, add HP StoreOnce to the list, and choose between that and ExaGrid. Right now, from my perspective these two companies are working with us closest on cross-certification and joint activities, while other do not seem to have much interest yet.

HP StoreOnce is cross-certified for full compatibility today, and ExaGrid already has new firmware beta that provides full application-aware integration - this should be released quite soon.

Thanks!

[Jpurdy79]

Thanks, I'll take a look.

[jamieorth]

We just did a comparison across all of the ones you are looking at, including the HP. We ended up on the Exagrid. The DataDomain was also nice, but pricey. We also have Backup Exec in our environment and the integration with that is awesome. We are *patiently* waiting on the full integration with Veeam, as of right now we backup to the utility share, which doesn't dedupe or replicate.

[Daveyd]

We were in the same boat as you. We evaluated Data Domain and Exagrid. We went with Exagrid for 2 main reasons. 1. Cost. 2. Expandability. We like the fact you could add additional devices to the "grid" on the Exagrid appliance rather than doing a forklift upgrade like DataDomain.

We purchased 2 Exagrid appliances about 8 months ago. One for our primary datacenter and one for our backup datacenter. It works really well with the Backup Exec intergration. Does a good job with replicating the data to the 2nd device across a 50MB link. However, the Veeam intergration is totally non existant at this time. As you know you have to have the Veeam backups reside on the utility share which does no compression or depuping along with NO automatic replication. We had to set up a manual script to copy the Veeam files from 1 location to the other. The Veeam backup files are 500GB in size (reversed incrementals with 7 days rollback). So copying 500GB over a 50MB link is very painful.

I was told that the Veeam intergration with Exagrid was slated for this month. Hopefully that is true as I am only backing up half our VMs using Exagrid as a target and my total VM backups (85 VMs with 7 days in 7 rollback in 7 jobs) is about 1.2TB in size

[Jpurdy79]

Gentlemen, I appreciate the input. I think we've narrowed it down between EMC Data Domain and the ExaGrid unit. Please keep me informed on the Veeam integration firmware update for the ExaGrid unit. I heard you can use the Veeam "source-side" deduplication in conjunction with the Exagrid "target-side" deduplication process. I'd love to hear some feedback.

[Daveyd]

Not sure if the mods in here can give an update on the availability of the Exagrid integration or not.

[Vitaliy S.]

Well.. Exagrid has recently announced that they are adding enhanced support for Veeam Backup and Replication. Here is the press release:
http://www.businesswire.com/news/home/2 ... eplication

As far as I know this firmware will be available in a few weeks. Hope it helps!

[mcrespi]

Hello all,

My name is Marc Crespi and I am the VP of Product Management at ExaGrid. We appreciate the interest in our product and are very excited about our partnership with Veeam.

I thought I would give you an update on the official availability of our fully integrated support for Veeam. We did pre-announce the shipment of the ExaGrid version that supports Veeam Backup and Replication on March 2nd. Our actual general availability date is slated for early April (if not a little sooner) at this time. While release dates can always slip (and it may) we would expect any delays to be very short. We are as eager as you to bring this exciting combination to market.

Should you have any questions regarding this, please feel free to contact me at mcrespi@exagrid.com.

Sincerely,
Marc Crespi
Vice President of Product Management
ExaGrid Systems

[Gostev]

Thank you Marc for chiming in and providing the update.

[Daveyd]

Awesome news! So, we have 2 ExaGrid appliances. One in our Primary DC and one in a DR DC. If we want to keep 7 rollbacks of all our VMs and have the target be our ExaGrid appliance at our primary DC and then replicate to our DR DC (when the firmware is released), what would the best backup type be? Reversed or "regular" incrementals?

[Gostev]

Regular.

[mvindell]

First time posting on these forums, I have been researching veeam for a couple of days now and have seen some webinars and such. Overall I am impressed and will most likely be utilizing it to backup and replicate our virtual environment. I have a quick question regarding deduplication on veeam vs deduplication on these appliances. Veaam comes built in with deduplication and compression, is there a need to purchase these appliances to perform that same function? Won't the block files already be deduped by the time the get to the target backup location?

[Gostev]

Technically speaking, it does make great sense to purchase these appliances to perform that same function for the 2 reasons mentioned below.

But of course, speaking from an end user perspective (I am doing personal research to gets tons of storage for my personal project), it all comes down to cost of specific solution, and individual consideration whether the purchase makes sense in your specific case. So, make sure that you perform ROI analysis of dedupe appliances against cheaper raw storage. For example, recently I learnt from other community members here that you can now get 32TB of raw iSCSI storage for under USD 7K. So, I recommend that you estimate what "raw" storage amount you will get with your dedupe appliance, and compare that with cheaper raw alternatives. Keep in mind that raw storage does has its own benefits (better performance), and that is important especially in lights of vPower based technology (which put your backup storage to work, making backup storage I/O performance quite important factor).

By the way, since I was recommending couple of specific dedupe appliances here, I must note that both of them are vPower ready. ExaGrid essentially provides vPower with raw storage perfomance (for most typical use cases), thanks to their unique architecture. HP StoreOnce has different architecture, but according to our joint testing it also provides acceptable performance for vPower functionality (for all possible use cases).

Anyway, 2 reasons to use dedupe appliance as Veeam backup target are as follows:

1. Veeam's dedupe is done on per-job basis. If you are planning on using multiple jobs, which most customers do because of job management considerations, dedupe appliance will help you to dedupe between those backup files.

2. Dedupe appliances have more effective dedupe, and can help you achieve additional storage space savings. Think about Veeam's dedupe as "1st pass" dedupe, it does help tremendously to speed up your backup speed and reduce backup window, but because it is done on the fly on 100MB/s plus data streams, we cannot really use advanced logic and very small block sizes (as it would kill processing performance and make backup window unacceptable). On the other hand, dedupe appliances are specially built and optimized to do fast and efficient dedupe, plus they have to deal with 5-10x less data (after Veeam's 1st level dedupe), so there is no problems for them using much better algorithms and much smaller block sizes. This makes it really powerful combo, either solution by itself is good, but when you get them together, you get the best of both worlds.

[Daveyd]

Regular.

Can you explain why a regular incremental would be better than a reversed incremental in my case? Also, if it recommended to do regular incrementals would I also need to do synthetic fulls and transform previous backup chains into rollbacks?

[Gostev]

Can you explain why a regular incremental would be better than a reversed incremental in my case?

Reversed incremental requires 3x I/O operations during backup. And dedupe appliances do not "like" I/O. This would reduce your backup performance noticeably. In fact, reversed incremental results in reduced backup performance even with raw disks, but not a significant as with smart storage.

Also, if it recommended to do regular incrementals would I also need to do synthetic fulls and transform previous backup chains into rollbacks?

No, transform is pointless with dedupe device, because it dedupes similar data between your multiple full backup files anyway.

[Daveyd]

I want to have 7 rollbacks and do forward incrementals to our replicating Exagrid appliances. So, Veeam would initially do a full backup creating a .vbk file. Then, every day for 7 days, the Veeam backup will create incremental vib files. After the 7th day, the oldest vib file will get overwritten by the most current vib file? So all that would really need to replicate is the vbk file (one time) and the vib files (every day)?

[Gostev]

After the incremental data run on the 7th day, new full backup (VBK) will be created synthetically using the previous 7 incremental backup files (VIB). This will start the new full backup chain. This does not mean however that whole new VBK will need to be replicated, I assume ExaGrid is smart enough not to replicate content of data blocks which are already present on the target ExaGrid device, meaning that replicating new VBK file will take absolute minimum of traffic. But you best double-check with ExaGrid on that.

[Daveyd]

So, it sounds like I have to choose to enable synthetic fulls when I create an incremental backup job, correct? When you say the new VBK will be created synthetically using the previous 7 VIBs, does that mean it will overwrite the initial VBK that was created? Will anything then happen to the existing 7 VIB files? Are they deleted all at once or are they overwritten once the next incremental kicks off?

[Gostev]

Correct. No, it will not overwrite, but instead create new full backup (read my previous response carefully). Previous chain will only be removed by retention when the latest incremental backup in that chain is no longer needed for restore.

[Daveyd]

Since it is creating a new vbk and not deleting anything until the last chain is no longer needed it sounds like I need to plan for double the storage space to hold 2 sets of vbks and vibs? Whereas reversed incrementals with 7 rollbacks only use and reuse 7 total files...6 vrbs and 1 vbk?

[Gostev]

No, double storage is not required to store 2 sets of VBK and VIBs. Look at the title of this topic

[Daveyd]

I am soory but I am really confused here. Must be the lack of sleep

I have want to be able to do a complete VM restore for up to a week. I create a forward incremental job, select 7 rollbacks and select to create a synthetic full every Sunday. So, the first backup runs on Sunday. It creates a vbk file. The next backups run Monday through Saturday each creating a vib file. So, now I have 6 vibs and 1 vbk say totalling 1TB. Sunday comes around again and a synthetic full is created. Now, this process creates another vbk file and I now have 2 vbk files that total 1TB plus whatever the new vbk file is. If I am correct so far, what exactly happens next on Monday-Saturday? Additional vib files are created each day leaving me with 2 sets of vibs (the current ones and the ones from the week before)? So, at the end of the 2nd Saturday the used space is 1TB + the new vbk + the new vibs?

Am I totally off base here? Do I need more sleep?

[Gostev]

Your description of the process is actually mostly correct, except your used space calculations. You are missing the fact that your target is deduping storage device. Thus, new VBKs will be taking little to no space. Remember, new VBK is created synthetically from data already available on device in previous VBK and VIBs.

[ccrichard]

I tested Veeam with lessfs on Fedora. Nice thing about it is it used SCP. I ended up going with Quantum dxi4510. The only weird thing I had to do is mount the smb/cifs share as and admin running in the cmd before I could have rights to the share from veeam, AND had to make a user on the dxi with the same credentials as the account that was running the veeam service.

Then I decided to get cute and backed up a physical server to the dxi. Not a bad idea, but I used rsync. Lots of little files are bad for dxi. It used up all the inodes, but I'm backup and running now that I use tar for the physical server.

In short, I've had good results from Quantum dxi4510 with Veeam.

[tasi]

Great thread.

I've been using Data Domain products for the past 3 years and they've been working out great. I've started to fill up the devices (and our support/maintenance is about to expire) and was looking to upgrade or find an alternative.

So it looks like ExaGrid is a good way to go for a dedupe device, I'll have to research it a little, but is the storage easily expandable, or is it the type of device you have to replace if you need more space?

If I were to consider raw storage, what products would Veeam or others recommend, which vendors work best with Veeam?

Thanks for any help.

[Gostev]

For raw storage, Dell Equallogic seems to be the one that is most often mentioned on this forum, and everyone seem to be quite happy about concerting performance, price, reliability and features. I have a feeling that nearly half of Veeam Backup customers with shared storage use Equallogic. At least it seems to be top choice lately, for sure. Thanks.

[tasi]

Thanks for the quick response Gostev.

Last question, what are most people using as a target, CIFS, NFS, iSCSI? What would typically give the best performance.

In my testing with Data Domain, I was assuming that NFS or using an NFS mount through the ESX server would work best, but found that using CIFS actually increased backup performance quite a bit.

[Gostev]

Block-based protocols (FC/iSCSI) are the single best choice for writing to backup target. To me, performance is secondary thing - it is reliability that is most important for BC/DR solution. And since transactional NTFS does not support network shares, using shared folder as your backup target increases the chance of backup data loss or corruption.

[Daveyd]

Great thread.

I've been using Data Domain products for the past 3 years and they've been working out great. I've started to fill up the devices (and our support/maintenance is about to expire) and was looking to upgrade or find an alternative.

So it looks like ExaGrid is a good way to go for a dedupe device, I'll have to research it a little, but is the storage easily expandable, or is it the type of device you have to replace if you need more space?

If I were to consider raw storage, what products would Veeam or others recommend, which vendors work best with Veeam?

Thanks for any help.

Exagrid is "grid" based. You add more appliances to the "grid" Unlike Data Domain.