Twinstrata vs. Amazon Storage Gateway

[ccitrano]

All,

I'm researching these two products along with Veeam Cloud Edition. I think Veeam Cloud Edition may not work for me because of the lack of the local cache.

Pricing aside, it looks like Twinstrata and Amazon function similar. Twinstrata offers different cloud providers, whereas Amazon is tied directly into their cloud. I think I'm considering the Gateway-Cached model for the Amazon side.

Does anyone have practical experience with these. I know everything is a function of Internet speed and capacity. I'm particularly interested in how folks are setting up their Veeam jobs and what style they are using (Incremental, Full Synthetics, Reverse Synthetics, etc.).

My goal is to have my Veeam Repository on local SAN dedicated to Veeam backups and somehow move data to the Twinstrata appliance or the Amazon Appliance. Or, is it best practice to run the Virtual Appliances using the SAN Storage (Veeam dedicated) and just have a single repository that has a really big cache???

Thoughts?
Chuck.

[dellock6]

Hi Chuck,
about the processing mode, usually forward incremental is the preferred method, since the vbk file is modified only at creation time, so the replica towards the cloud provider happens only once, while with reverse incremental you need at least to hack the registry to have e fixed file name, and check if the replica software can do byte level replica. If it only does file level, every day the reverse vbk file will be uploaded completely.

Another possible storage gateway can be a local storage server based on Windows, and a replica software like Veeam Cloud Edition on board that can replicate data to several cloud providers. I think it can be maybe cheaper and more flexible to configure.

Luca.

[mongie]

We have twinstratas... I found that they *bareley* work. From memory, the amazon appliance doesn't dedupe. The cloud array is meant to dedupe, but it hasn't helped me in practice.

I was recently testing cloud restore, and while I havent been able to get the cloud array working, I had the was gateway up and running in about 30mins. Much easier to use in my opinion.

You should look at the riverbed whitewater as well. Apparently, they get awesome dedupe, but I think they're a bit more expensive.

Also, make sure you have sufficient bandwidth... That seems to be my major issue most of the time, especially since I get no dedupe, so my weekly fulls get uploaded at full size.

[ccitrano]

Mongie,

thanks for the reply. Can you provide any more details about Twinstrata? Price aside, the story sounds real nice and it seems that a properly configured appliance with a large cache may be a good solution. When you say barely ... does that mean you guys were constrained by cost or function??

With regards to deduping ... the Twinstrata guys said that they rely on Veeam deduping and turn off theirs ... I think. In this case it seems the same amount of data would be moved off with regards to the Amazon solution.

A lot of this seems to boil down to the amount of actual data that moves off your network. Time, Bandwidth and Size are all interelated. It get's confusing in my mind trying to figure out who is doing the best job of moving the smallest amount of data off the network. Looking for a vendor with either tight Veeam integration or some other magic sauce.

I'm going to try one of these solutions to play with and I'm sure things will become pretty apparent. Experience is probably the best teacher here and I'm really interested in hearing from folks who have implemented one of these solutions ... successfully or not.

Cheers.
Chuck.

[veremin]

As to the Twinstrata implementation, it might be worth reviewing this topic or this document which is primarily related to VB&R, CloudArray best practise.

Hope this helps.
Thanks.

[mongie]

Hi Chuck,

Having re-read my original post, I guess I'll try and be a bit more specific, since I don't think it was clear what I was trying to say.

Using one of my sites as an example... We generate ~ 1TB of FULL backups per week, and around 200GB of incrementals. The site has a 20Mbit internet connection and is uploading to Amazon AWS. The TS appliance is currently just treading water... basically uploading dirty data all week and finishing just before we have the next lot of fulls run.

There are two issues that I have to deal with at this site.

1. Lack of CloudArray Deduplication (even though there is an option when configuring it for Compression, and an option for Dedupe) leading to larger uploads.
2. Lack of Internet Bandwidth

You wouldn't think that lack of bandwidth would be the fault of the twinstrata, but if the appliance actually performed deduplication of data, you would see much lower uploads to the cloud and much lower amounts of storage used in the cloud. Veeam doesn't really help in this instance... yes the backups are smaller than they would be without compression or deduplication, but multiple full backups are still uploaded at full weight.

My understanding of the biggest competitor (certainly at the time we bought our appliances) Riverbed Whitewater is that it produces great deduplication, meaning that it can inteligently choose whether it needs to re-upload duplicated blocks to the cloud, saving money in storage and reducing the requirement for higher bandwidth pipes.


I guess this is a business decision as to whether it matters... but it would certainly matter to me.

My two sites with TwinStratas currently running are using ~ 10TB of cloud storage. 5.2TB at .13c/GB and 4.2TB at 0.8c/GB. Thats not SUPER expensive, but its also not SUPER cheap.

[ccitrano]

Mongie,

Thanks ... this is really useful information.

Can you dive a little deeper on one aspect for me. I was drawn towards the companies that provided a local cache because my thinking was that disk latency would not be introduced until the cache was full. Using Twinstrata as an example, I could create a Virtual Appliance with 10 TB of cache so that my Veeam B&R software would just target it and the entire backup set for me would fit on the cache ... hence keeping it local. I understand that somewhere behind the scenes, the Twinstrata would have to move the data on the cache into the permanent cloud storage. My current backup set is 8TB of Veeam files that are using Veeam dedup and compression.

I'm trying to get it sorted out in my mind about any quirks that are introduced if TS doesn't keep up because of bandwidth. Does it eventually drown ... or cause my Veeam backups to fail?

I'm currently doing my backups using reverse incrementals ... which I am switching to forward incrementals. I'm thinking of doing synthetic fulls once a week so I would take the big hit then.

I haven't looked at Riverbed yet. I'll give them a read. I have read some articles that the Amazon gateway will challenge these types of gateways because they cut out the middleman. I guess the middleman gives you more backend storage options ... but that's not a huge concern for me.

Cheers.
Chuck.

[mongie]

Yes, if you have a large local cache, you do not really see any limitation on Veeam writing backups to disk (until the cache is full).

If the TwinStrata can't keep up (with uploading data to the cloud) you don't really notice a performance hit, but you also have no off-site protection.

With one of my two sites, there is something like 7TB of dirty data in the cache that is never going to be finished uploading.


Amazon Storage Gateway is very nice, but there are two issues...

a) No deduplication (that I'm aware of)
b) Subscription only ($125/month IIRC)

I found it WAY easier to use than the CloudArray. It just works!

[dellock6]

Another solution could be to create your own "software defined" local cache. For example, a Windows Server with Veeam Cloud Edition onboard (or even Cloudberry, since Veeam is based on this product). In this way you are free to size local cache and other parameters at your will.

Luca.

[andyboy]

We've been using Twinstrata for a while now. First, we purchased the Virtual Twinstrata appliance and had a 700GB dedicated cache to it. Each night our backups will write to that cache and then get flushed out to our Amazon cloud as data get written (nothing is stored locally). This worked great for a while until our ESX environment started to grow. The problem was that the Twinstrata cache would fill up faster then flushing out (we have a 300MB bandwidth), making our backup jobs run slow. The other issue was also restoring VM's. When we needed to restore a VM, veeam has read all that data from the cloud and retrieve it, which took a while. One more concern was that we wanted to keep a local copy of our backups on site and off of our SAN so we went ahead and purchased the physical appliance. The physical appliance allows us to have a local copy of our back ups and also in the cloud. Our backup jobs are now faster, stored locally and offsite. Mind you that we are using Revers Incremental and Veeam is doing all the dedup. Veeam does a fantastic job in dedup so there's no reason to get a dedup appliance.