Off-siting 30-40TB - any recomendations?

[Yuki]

Hi All,
We have 2TB worth of server backups and about 30TB of raw data that can be stored either on a NAS or on Server (this will depend on our off-siting solution). Most of this data is an archive type that doesn't need to be accessed. We are going with a multi-tier backup solution. First tier is a duplicate server/storage system in house that is receiving Veeam backups and replicas. This will protect from an isolated in building fire, flood, or other similar small scale events. However, the second part is what we haven't figured out yet - major disaster protection mechanism in case of the complete destruction of the building and all assets.

Wile we can find a solution for the 2TB worth of server systems, we are having hard time finding a system for the 30TB+ of archive data. Ideally we would move any data not accessed within last 6 month onto a NAS or a folder on a server and have it be picked up and uploaded into an internet service like Amazon Glacier. But we are not aware of a reliable, enterprise grade software to keep track and upload all of the data to Glacier.

We could even possibly go with Amazon S3 or others, but would like some recommendations on software and possibly hardware (if needed). I know a few companies are offering backups to S3, but most of them have limitations.

We used tapes before, but we would really like to move away from them for a number of reasons. I don't want to debate those reasons right now - we can always go back to tape if we don't find a viable internet based solution.

Any recommendations from those who have done it? Any example of particular setups would be appreciated.

[dellock6]

I'm testing a possible and easy way to upload backups to Amazon Glacier, since it has no client or direct interface like FTP or rsync but only API. I probably found a good solution, only need some more tests to validate it. I'll update this thread with news as soon as I will finish.

Luca.

[Yuki]

Hi Luca,
any chance to give at least a hint of what you are doing so maybe i can research it in parallel?

We are not looking for just the lowest cost, but are interested for a good all-around system (reliable and dependable).

[dellock6]

Ok, I'm using CloudBerry S3 Backup Server Edition on top of the Veeam Repository server. It supports AWS Glacier, plus many other features like run as service, scheduling, bandwidth throttiling, encryption (nice to ship files to Glacier and be sure they cannot be open if lost/stolen) and many other if you look at it.

I'm doing a complete set of tests like running it as post-veeam job, selective backup, and others. Oh, and collecting all the findings in a future blog post

Luca.

[cparker4486]

Ok, I'm using CloudBerry S3 Backup Server Edition on top of the Veeam Repository server.

Can you share your findings so far? I've got the same product and am looking to do the same thing you appear to be researching.

What I'm not sure about though is where to do the encryption. I don't like the idea of having my data locked into a proprietary format e.g. utilizing Cloudberry's encryption. I'm thinking about doing something like once a month copy the VBK file to a temporary location, encrypt it with TrueCrypt, and then upload it with Cloudberry.

Another thing I don't find too appropriate for my usage is their Advanced mode because it would require using a Cloudberry product to access the data after it's been uploaded.

Anyway... I've got a lot of thoughts spinning around in my brain so I'd be very interested to hear your thoughts on this.

[dellock6]

Right now I'm only testing CloudBerry as a mean to ship data into Glacier, since I haven't found any other tool to map a Glacier's vault as a local CIFS/NFS/FTP server. Probably solutions like Twinstrata could in the future add Glacier support to their storage appliances, but I like the idea of having a software-only tool in my control. Probably with better coding abilities I would have used Glacier APIs directly into the Windows VM holding Veeam...

Yeah, CloudBerry encryption is undesirable, but the process Veeam -> Disk -> Truecrypt -> Glacier sounds long and complicated. But I haven't tried so far cloudberry encryption.

Luca.