Discussions related to using object storage as a backup target.
Post Reply
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Immutable backup to Amazon S3?

Post by HendersonD »

I just engaged Amazon in a conversation about backing up to Amazon S3. We already do backup to a DR site but I am interested in backing up to Amazon for one reason, the ability to do immutable backups. In other words, a backup someone trying to infect us with ransomware cannot erase. It seems that Amazon offers a bewildering number of ways to use it as a backup target, none of which I am familiar with at the moment.
  • AWS Storage Gateway, Tape Gateway, File Gateway, and Volume Gateway
  • S3 Standard
  • S3 Intelligent Tiering
  • S3 Infrequent Access
  • S3 Glacier
  • S3 Glacier Deep Archive
Is anyone backing up to Amazon S3 for just immutable backups and can give me some advice on how this all works?
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Gostev »

v10 support immutability natively with the following S3 storage classes:
  • S3 Standard
  • S3 Infrequent Access
  • S3 One Zone Infrequent Access
Backup to object storage is a feature of the scale-out backup repository Capacity Tier.
Immutable backups option can be enabled when registering an S3 bucket.

Thanks!
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

Gostev,
Thanks for the quick response. Any thought about offering immutable backup with S3 Glacier or S3 Glacier Deep Archive? As you can imagine, I am asking since these two Amazon tiers are less costly than S3 One Zone Infrequent Access
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Gostev » 1 person likes this post

These S3 tiers are very special in the way they function, as they are built to be a tape replacement for the "write once read never" use case, which makes them perfect for data archival. But they are not compatible in principle with the Capacity Tier paradigm, which is designed to provide transparent on-prem backup storage extension.

So while we ARE working on integrating with these tiers, it is important to understand that the Capacity Tier will never support them. You will be able to use these only as a part of the dedicated Archive Tier of the scale-out backup repository. But, it's not something we're ready to announce details on just yet.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

I talked to Amazon again today and besides natively supporting Veeam for the three tiers that Gostev mentioned in his previous post, they also said that I could setup some type of gateway at my end that basically sets up Amazon S3 as a virtual tape library. With this setup I could then write to S3 Glacier or S3 Glacier Deep Archive.

Is this even an option to consider? The people I am working with at Amazon have been great but I get the impression that they are not that familiar with Veeam and how it interacts with Amazon S3
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutable backup to Amazon S3?

Post by dalbertson »

Woo Hoo...my time to shine :)

https://www.veeam.com/wp-using-aws-vtl- ... guide.html

You can use the SGW in VTL mode but its a tape job and not part of a scale out repo. But it does meet the objective of getting data into deep archive.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

Can I use the new immutable backup feature of V10 with this setup? I am really looking to use S3 because of this one feature. I have read too many reports of ransomware attacks happening right after the perps erase all backups. I want to make sure I have at least one backup to S3 that cannot be erased not matter what
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutable backup to Amazon S3?

Post by dalbertson »

You can’t use this with the v10 immutability feature. But this is the OG airgap method. This method is offline and stored in glacier. When your tape job is done it ejects and exports the tape and then it’s moved to glacier. You get it back you have to manually recall the tape from AWS console. This method protected from random ware before object lock was around.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

So here is what I have so far. I can use the v10 immutable feature with these Amazon tiers
  • S3 Standard
  • S3 Infrequent Access
  • S3 One Zone Infrequent Access
This is easier to setup than the gateway option according to Amazon

I can use the gateway option with these tiers which are cheaper than the three mentioned above
  • S3 Glacier
  • S3 Glacier Deep Archive
This is a bit more difficult to setup than the first option

Are there other things to consider in trying to decide which way to go?
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutable backup to Amazon S3?

Post by dalbertson »

Those are your choices. Both are not that difficult to setup if you understand the design. The difference between the two will be job type and strategy.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

Will Veeams retention policy work against the S3 backups? In other words, if I have Veeam set to a 30 day retention when backing up to S3, it will add incrementals correctly but will it also delete incrementals to maintain the 30 days of backup? If not, is this setup within S3?
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutable backup to Amazon S3?

Post by dalbertson »

It does. The backup job retention setting will also remove the date from S3 when met.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Immutable backup to Amazon S3?

Post by HendersonD »

Cool, thanks for the assistance. I will keep marching forward with this project
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutable backup to Amazon S3?

Post by dalbertson »

No problem. If you run into any more questions or issues you know where to come :)
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
ndconway
Novice
Posts: 3
Liked: never
Joined: Sep 27, 2021 9:22 pm
Full Name: Norman
Contact:

Re: Immutable backup to Amazon S3?

Post by ndconway »

Frustrated!!

I've now engaged an AWS paid professional that states you cannot write VM backups directly from our local Linux server to cloud S3 storage using Veeam (as we were originally told). He maintains we can only create immutable backups locally and then convert the files to blobs and copy them up to S3 storage in the cloud. Once copied, up you cannot browse the contents of the files to pull down selected data in the event of a ransomware attack.

Is there any chance, I've been mislead and we really can write our initial backups directly to the cloud on the first-pass and get file level access once in the cloud? It seems like the process should be straight forward and manageable, but so far it's been anything but. Help!
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Gostev »

May I ask why are you even talking to an AWS paid professional about Veeam, as opposed to a paid Veeam professional? :D

Writing VM backups directly from your local Linux repository server to cloud S3 storage using Veeam is exactly what most of our users do. Once copied up, all the same restore types are still available from backups in S3 (including browsing the contents of the files to pull down selected data in the event of a ransomware attack). You can even do an instant VM recovery and run a VM off of its cloud backup!
Mildur
Product Manager
Posts: 9848
Liked: 2607 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Mildur »

Once copied, up you cannot browse the contents of the files to pull down selected data in the event of a ransomware attack.
You don‘t need this feature. If there was a ransomware attack, connect the aws bucket to a newly installed veeam backup server and begin todo your restores. It‘s a really simple process.

we really can write our initial backups directly to the cloud on the first-pass
Veeam is working on such a feature (teased on VeeamOn 2021). But it‘s not available today.
Product Management Analyst @ Veeam Software
ndconway
Novice
Posts: 3
Liked: never
Joined: Sep 27, 2021 9:22 pm
Full Name: Norman
Contact:

Re: Immutable backup to Amazon S3?

Post by ndconway »

Gostev: Thanks for your support. If you look at the posts that just came in on this subject. Unless I'm reading it wrong, your post conflicts with Mildur's who seems to indicate the feature is not out yet. We're running V11 and I was hoping we could get this process completed soon. Can you recommend someone to help us?
Mildur
Product Manager
Posts: 9848
Liked: 2607 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Mildur »

Today, You can offload restore points from your linux backup repo directly to the s3 bucket with a SOBR.

But you cannot backup a linux agent directly in to the s3 bucket.

That was I was referring too. :) I am Sorry, if there was a missunderstanding.
Product Management Analyst @ Veeam Software
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutable backup to Amazon S3?

Post by Gostev »

There's no conflict between my and Mildur's posts.

I am talking about the following scenario: local backup to Linux repository + copy of these backups to cloud S3 storage.
This is the best practice approach because it meet the 3-2-1 rule of backups, while providing fastest backup and restore performance.

Mildur however is talking about direct backup of production data to cloud S3 storage.
This has limited use because it does not meet the 3-2-1 rule, impacts production environment with long-living VM snapshots, and is just slow for both backups and especially restores.

As for Veeam professionals, see here > https://www.veeam.com/find-a-veeam-accr ... rtner.html
ndconway
Novice
Posts: 3
Liked: never
Joined: Sep 27, 2021 9:22 pm
Full Name: Norman
Contact:

Re: Immutable backup to Amazon S3?

Post by ndconway »

Thanks. We already do non-immutable backups at the primary site with replications coming to the cold site. We wanted to add in immutable back ups directly to the cloud without the copy feature. Thanks for the clarification.
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests