Remote backup to AWS Glacier

[dellock6]

Has we talked about it in other threads, there seemed to be some sort of interest about sending long-retention backups to Amazon Web Services "Glacier".
As promised, I published yesterday a short tutorial about it:

Freeze your backups into AWS Glacier

As always, feedback is welcome, here or on my blog.

Thanks,
Luca.

[chrisdearden]

Its an interesting concept - might be an idea to do some cost calcluations on how much that storage will cost and more importantly how much it will cost to recover from ?

[veremin]

My two cents regarding this new service.

From my perspective, the Amazon Glacier should be only used as a long-term storage for incredible amount of data which stays unclaimed for long period of time; by the means of AG you can unload local storages of big enterprise, etc.

Futhermore, i do believe that it was deliberately created to meet exactly such expectations and no other, as a reason of it we have a variety of limitations such as period of time required for getting information from the cloud (3-5 h.), fact that only 5% of your data can be downloaded per month for free, costs for downloading information and so on.

So, it’s not something very interesting and affordable if your goal is 24/7 availability of your information, or you’re looking for a remote source for storing high-claimed data and so on. In contrast, if you’re eager to find more or less cheap remote storage for saving huge amount of information, which is likely to be unclaimed for a long time, amazon glacier is exactly what you need.

Actually, It's quite an intresting discussion, since only recently has this service been represented to general public, and i'd like to hear pros/cons of using it, or probably some commentary from the persons who have already implemented it in their environment.

[dellock6]

That's correct, and is what I wrote in my post: Glacier has some limitations written right to limit the downloads. Seems like it's more an imposed limit and not a technological one, just like asking you to pay for downloads but not for uploads.

It's to me the last tier of a complete backup strategy, and I already have a couple of customers using it (that's where the idea for the post came from): they both had law compliance about long term retention, they had to keep all data available for 10 years for audits only, not because users would ask for restores.
With Glacier, you can get rid of tape libraries, tape storage, maintenance and other tape-related activities.

If you need instead a remote backup service in a second tier (supposing local storage for veeam is the first tier, and Glacier the third) there are other services around where downloads are not limited. Keeping AWS example, I would use S3 as a secondary tier and Glacier for long-term retention.

Luca.

[stevenrodenburg1]

As Glacier is not meant for, and designed for high-speed restores, the only use-case i can think of is of archiving of backups that are not likely to be restored (like monthly or yearly backups). But as a whole, i don't regard Glacier as a viable option for backup-storage. If you read Amazon's documentation, they almost seem to steer customers away from (mis-)using it as a backup storage-platform. Retrieval is a main issue but i don't want to repeat what v.Eremin already wrote.

[veremin]

Besides payment, other limitations regarding download, such as the amount of time required for getting information (3-5 h.) from Glacier, seem to be more technological ones.

As far as I’m concerned, Amazon Glacier’s architecture is based on using so-called streamers (tape storages). Although I’m not familiar with exact internal structure of Glacier, I can assume that it something like huge number of common HHDs, used for cashing uploaded and downloaded data, and multitude of streamers, which are playing the role of long-term storage.

So, when you make request for receiving data, some kind of a robot chooses appropriate tape and put it into the streamer, after that data is transmitted to HDD, from which you’ll eventually get it. Long period of time necessary for such procedure is caused by overload of these robots and by the fact there are numerous people willing to get their information, as well as, there are numerous tapes.

Basically, it’s a time required for taking ownership over the robot, plus a time needed for robot to complete procedure.

[dellock6]

It could also be a "political" decision. Maybe Glacier can possibly offer vaults download with lower waiting times, but this would eat market over S3 storage. Also the pricing is designed to make you easily upload data, but not to download it.
I also read around about some custom designed disks able to run at really low speed, to be shut down completely when not in use and other stories, whatever they are Glacier is a great product, "IF" we understand first its use case.

Luca.

[hmusa]

Is anyone using Dropbox, Google Drive or Amazon services to sync their Veeam backup files offsite? Set the Veeam destination folder to one that's synchronized to either - instant offsite backups. If this is stupid, it's probably on the bandwidth and security level. Sorry if this has been discussed, but I didn't find it in searching. Still looking for a nice way to offload our <1TB backups / nightly ~50GB. Last time I checked, Amazon AWS Storage Gateway was prohibitively expensive on the bandwidth metering side. Thx.