Comprehensive data protection for all workloads
Post Reply
Yuki
Veeam ProPartner
Posts: 252
Liked: 26 times
Joined: Apr 05, 2011 11:44 pm
Contact:

Off-siting 30-40TB - any recomendations?

Post by Yuki »

Hi All,
We have 2TB worth of server backups and about 30TB of raw data that can be stored either on a NAS or on Server (this will depend on our off-siting solution). Most of this data is an archive type that doesn't need to be accessed. We are going with a multi-tier backup solution. First tier is a duplicate server/storage system in house that is receiving Veeam backups and replicas. This will protect from an isolated in building fire, flood, or other similar small scale events. However, the second part is what we haven't figured out yet - major disaster protection mechanism in case of the complete destruction of the building and all assets.

Wile we can find a solution for the 2TB worth of server systems, we are having hard time finding a system for the 30TB+ of archive data. Ideally we would move any data not accessed within last 6 month onto a NAS or a folder on a server and have it be picked up and uploaded into an internet service like Amazon Glacier. But we are not aware of a reliable, enterprise grade software to keep track and upload all of the data to Glacier.

We could even possibly go with Amazon S3 or others, but would like some recommendations on software and possibly hardware (if needed). I know a few companies are offering backups to S3, but most of them have limitations.

We used tapes before, but we would really like to move away from them for a number of reasons. I don't want to debate those reasons right now - we can always go back to tape if we don't find a viable internet based solution.

Any recommendations from those who have done it? Any example of particular setups would be appreciated.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Off-siting 30-40TB - any recomendations?

Post by dellock6 »

I'm testing a possible and easy way to upload backups to Amazon Glacier, since it has no client or direct interface like FTP or rsync but only API. I probably found a good solution, only need some more tests to validate it. I'll update this thread with news as soon as I will finish.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Yuki
Veeam ProPartner
Posts: 252
Liked: 26 times
Joined: Apr 05, 2011 11:44 pm
Contact:

Re: Off-siting 30-40TB - any recomendations?

Post by Yuki »

Hi Luca,
any chance to give at least a hint of what you are doing so maybe i can research it in parallel?

We are not looking for just the lowest cost, but are interested for a good all-around system (reliable and dependable).
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Off-siting 30-40TB - any recomendations?

Post by dellock6 » 2 people like this post

Ok, I'm using CloudBerry S3 Backup Server Edition on top of the Veeam Repository server. It supports AWS Glacier, plus many other features like run as service, scheduling, bandwidth throttiling, encryption (nice to ship files to Glacier and be sure they cannot be open if lost/stolen) and many other if you look at it.

I'm doing a complete set of tests like running it as post-veeam job, selective backup, and others. Oh, and collecting all the findings in a future blog post :)

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
cparker4486
Expert
Posts: 231
Liked: 18 times
Joined: Dec 07, 2009 5:09 pm
Full Name: Chris
Contact:

Re: Off-siting 30-40TB - any recomendations?

Post by cparker4486 »

dellock6 wrote:Ok, I'm using CloudBerry S3 Backup Server Edition on top of the Veeam Repository server.
Can you share your findings so far? I've got the same product and am looking to do the same thing you appear to be researching.

What I'm not sure about though is where to do the encryption. I don't like the idea of having my data locked into a proprietary format e.g. utilizing Cloudberry's encryption. I'm thinking about doing something like once a month copy the VBK file to a temporary location, encrypt it with TrueCrypt, and then upload it with Cloudberry.

Another thing I don't find too appropriate for my usage is their Advanced mode because it would require using a Cloudberry product to access the data after it's been uploaded.

Anyway... I've got a lot of thoughts spinning around in my brain so I'd be very interested to hear your thoughts on this.
-- Chris
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Off-siting 30-40TB - any recomendations?

Post by dellock6 »

Right now I'm only testing CloudBerry as a mean to ship data into Glacier, since I haven't found any other tool to map a Glacier's vault as a local CIFS/NFS/FTP server. Probably solutions like Twinstrata could in the future add Glacier support to their storage appliances, but I like the idea of having a software-only tool in my control. Probably with better coding abilities I would have used Glacier APIs directly into the Windows VM holding Veeam...

Yeah, CloudBerry encryption is undesirable, but the process Veeam -> Disk -> Truecrypt -> Glacier sounds long and complicated. But I haven't tried so far cloudberry encryption.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Post Reply

Who is online

Users browsing this forum: ante_704, Bing [Bot], mibrown9954, mrmccoy007 and 323 guests