Discussions related to using object storage as a backup target.
Post Reply
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

AWS S3 bucket configuration

Post by stewsie »

Hi

I have seen the Veeam documentation regarding using S3 storage. Does anyone have information on how to configure the S3 bucket? Specifically interested in the access requirements? Suspect a role needs to be created with permission to the bucket?

I am an AWS novice so please excuse my basic questions

Thanks
wishr
Veteran
Posts: 3077
Liked: 455 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: AWS S3 bucket configuration

Post by wishr » 1 person likes this post

Hi Paul,

Speaking about general S3 configuration, please refer to the official AWS guides: Setting up Amazon S3, Creating a bucket. For the list of permissions required to use S3 with B&R, please take a look at the Required permissions section of the B&R User Guide.

Thanks
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

Re: AWS S3 bucket configuration

Post by stewsie » 1 person likes this post

Good starting point. Thanks
chris.arceneaux
VeeaMVP
Posts: 690
Liked: 372 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Georgia, USA
Contact:

Re: AWS S3 bucket configuration

Post by chris.arceneaux » 1 person likes this post

Also, if you're looking for quick setup of the S3 bucket, I've created AWS CloudFormation Templates to build out everything you need in AWS automagically:
https://github.com/VeeamHub/veeam-aws-cloudformation
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

Re: AWS S3 bucket configuration

Post by stewsie »

Hi

I now have an S3 bucket ready to try the immutable backups. Is it possible to configure the capacity tier to only be used by specific Veeam jobs? I need to test this first before confirming this is the right solution so I do not want all Veeam jobs being moved to this capacity tier.

Also when backup or copy jobs are moved to the capacity tier is a full created so there is no dependency on the backup chain on disk? If not how will the chain work if backups are compromised on site but not S3?

Thanks
dalbertson
Veeam Software
Posts: 492
Liked: 175 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: AWS S3 bucket configuration

Post by dalbertson »

Hi @stewsie. You can not easily limit which jobs are offloaded to capacity tier on a SOBR. You could create a new SOBR and point only the jobs you want to offload to that new SOBR.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

Re: AWS S3 bucket configuration

Post by stewsie »

That was what I suspected.
Thanks
Gostev
Chief Product Officer
Posts: 31746
Liked: 7250 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: AWS S3 bucket configuration

Post by Gostev »

Needless to say, the new SOBR can be backed by the same exact storage device as the existing one... just a different folder on the same volume. Think about SOBR as a storage policy.
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

Re: AWS S3 bucket configuration

Post by stewsie » 1 person likes this post

I created a new SOBR and have created the capacity tier. Now to carry out testing and read more about this especially the recovery options.

Thanks all
stewsie
Veteran
Posts: 281
Liked: 24 times
Joined: May 22, 2015 7:16 am
Full Name: Paul
Contact:

Re: AWS S3 bucket configuration

Post by stewsie »

Not a question about the bucket configuration but just wanted to know the protocol used by Veeam to upload data to S3 and the security of the transfer of data. Cannot see anything mentioned about this. Thanks
chris.arceneaux
VeeaMVP
Posts: 690
Liked: 372 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Georgia, USA
Contact:

Re: AWS S3 bucket configuration

Post by chris.arceneaux » 1 person likes this post

Hi Paul,

This topic is discussed in this forum thread.

Here are some key points:
  • All Veeam interaction with the Object Storage provider is done using the provider's RESTful API which uses the their SSL/TLS certificate to create a secure connection.
  • Veeam Backup & Replication has the ability to encrypt Veeam backups at the Backup Job-level and the Capacity Tier-level
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: AWS S3 bucket configuration

Post by frankive »

@chris.arceneaux
I have been using your cloud stack template and it has always created the bucket in Stockhom-region (which I want).
When I run it now it creates the bucket in USA.

Is there something I am doing wrong now?
oleg.feoktistov
Veeam Software
Posts: 2010
Liked: 669 times
Joined: Sep 25, 2019 10:32 am
Full Name: Oleg Feoktistov
Contact:

Re: AWS S3 bucket configuration

Post by oleg.feoktistov »

Hi Frank,

Region of bucket creation would depend on the region you launched your CloudFormation stack from.
Might it be the case that default region in your aws cli client config has been changed?

Thanks,
Oleg
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: AWS S3 bucket configuration

Post by frankive » 1 person likes this post

Thanks for heads up, that did the trick!
lando_uk
Veteran
Posts: 377
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: AWS S3 bucket configuration

Post by lando_uk »

Hi,

Just checking, is this stack still valid and good for 2023?
veremin
Product Manager
Posts: 20360
Liked: 2287 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: AWS S3 bucket configuration

Post by veremin »

Based on the recent updates, it seems to be valid. However, let's summon @chris.arceneaux for further confirmation. Thanks!
chris.arceneaux
VeeaMVP
Posts: 690
Liked: 372 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Georgia, USA
Contact:

Re: AWS S3 bucket configuration

Post by chris.arceneaux »

It is indeed valid.

This can be confirmed by comparing the permissions required listed in our documentation with the permissions the sample CloudFormation template (standard/immutable) is configuring.
lando_uk
Veteran
Posts: 377
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: AWS S3 bucket configuration

Post by lando_uk »

Hi Chris
I ran this on our test yesterday, all good, but AWS Config is complaining about the user, it fails the following audit:

[IAM.2] This AWS control checks that none of your IAM users have policies attached. Instead, IAM users must inherit permissions from IAM groups or roles.
https://docs.aws.amazon.com/config/late ... check.html

It's not really an issue and can be fixed, but it would be great of the cf script was changed to remedy this. eg, create the role, add user to role?
chris.arceneaux
VeeaMVP
Posts: 690
Liked: 372 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Georgia, USA
Contact:

Re: AWS S3 bucket configuration

Post by chris.arceneaux » 1 person likes this post

Hi Mark,

Ah yes...I see. I'll look into this but can't provide an ETA. I'dd add that Pull Requests are welcomed to all VeeamHub repositories.
lando_uk
Veteran
Posts: 377
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: AWS S3 bucket configuration

Post by lando_uk »

Hi again.
The bucket is also failing these checks:
S3 buckets should have server-side encryption enabled
S3 buckets should require requests to use Secure Socket Layer

?
chris.arceneaux
VeeaMVP
Posts: 690
Liked: 372 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Georgia, USA
Contact:

Re: AWS S3 bucket configuration

Post by chris.arceneaux » 1 person likes this post

The CloudFormation template provided is an open source sample to be used when creating an AWS S3 bucket for use with Veeam.

As you've seen, scanners like AWS Config can sometimes find additional items to configure. I recommend testing the change in your environment. If the changes work for you, feel free to submit a Pull Request to this GitHub project with the required updates.

As mentioned previously, I'll look into this but cannot provide an ETA.
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests