9.5 Update 4 and Amazon S3

[stefan1967]

Hello,

could you explain me the detailed process for restoring data from an S3 storage after disaster (all local performance tiers are gone, only capacity Tier in S3 with backup data is available).
I tried to restore but I got a message, that data from the local performance Tier are missing (I deleted all local backups for test the DR Szenario).

Many thanks!
Stefan

[Gostev]

Did you do SOBR rescan before doing the restore?

I know it will work if you create new SOBR and add your object storage as Capacity Tier. You will then be asked if you want to import existing backups from object storage. This is the main "all is lost" scenario that we tested.

But since you still have the old SOBR, I wonder if simple SOBR rescan will restore stubs on Performance Tier. I need to test that!

[veremin]

If all extents are gone, you will need to add at least one fresh extent to which backup metadata can be restored.

So, in your case what can help is:

- Removing lost extents from SOBR
- Adding new extent to SOBR
- Re-scaning SOBR

After that you should be able to execute restore process.

Thanks!

[Gostev]

He does not have any extents lost, just backups deleted... so, just rescan and that's it?

[veremin]

It's not clear from the description, actually:

all local performance tiers are gone

Anyway, in this case re-scanning SOBR should be sufficient.

Thanks!

[anthonyspiteri79]

Just as a heads up, there are a few of us internally working on a Cloud Tier Deep Dive White Paper which will contain explanations around scenarios like this. We hope to have it out in 4-6 weeks.

For Stefan, I can say that I have validated the process a number of times now and it works as expected. If you are still having issues feel free to reach out.

[GreenAlpha55]

Fantastic to hear that 'copy' will be made available for the Capacity Tier in the next update.

Once GFS on backup jobs is added our backup architecture will be extremely simplified. Simple is always the golden key.

Then going forward I will be doing;
Production Storage > Backup Job w/GFS > Capacity Tier copy & move

[Gostev]

Frankly speaking, I am really excited about this stuff myself

[wjching]

Hi Guys,

Have a question regarding this as well.

When configuring this SOBR for storage tiering with S3. There is an option you can specify pertaining the age of your backup before it is being pushed to S3. Just curious, as I defined that as "0" days, how soon before it begins pushing to S3? I was expecting an instantaneous push from local SOBR to S3, but that didn't happen.

Also I suppose using PowerShell command to override and push it on a manual basis can be helpful for this scenario.

Thanks in advance for any help !

Regards,
Wen Jun.

[anthonyspiteri79]

Hey there Wen.

There are two conditions that need to be met for the data to be offloaded. First is if the backups are outside of the operational restore window as dictated by that policy value.

The other is if the backup chain is sealed. Have a read of this to get a better idea https://helpcenter.veeam.com/docs/backu ... l?ver=95u4

[wjching]

Hi Anthony,

Appreciate the elaborate explanation regarding SOBR Offloading to S3. Based on the provided resource link. I think this method is not ideal for customer's who want to DR to AWS given that they have a requirement of 1 Day RPO and RTO, and that they are running a Forever Incremental Backup Chain. Not unless they run full Synthetic Backup on a Daily Basis followed by a SOBR offload to S3, which i think is a little overkill given the circumstances. Any idea when will Veeam allow for direct backup to S3 without going through SOBR ? For Example, running a backup copy job to S3.

Thanks again for any advice or suggestions

Regards,
Wen Jun.

[anthonyspiteri79]

Hey there Wen.

The Cloud Tier isn't intended to be a DR solution...it's about offloading data from what is more expensive storage, to cheaper storage. There is only every one copy of the data that is either on the Performance Tier, or the Capacity Tier.

In terms of running a Synthetic/Active full...out of interest, what issues do you foresee with that? Also wondering what your overall use case is?

We can't comment on future features around the Cloud Tier just yet, but the direct backup to S3 has been requested previously.

[dariusz.tyka]

Hi all,

have a question regarding adding Amazon S3 object storage. To do that you of course need a cloud credentials. I make a research and also asked support (ID# 03413397) but got info that there is no document explaining what are the minimum account permission to access single S3 bucket. I made some tests and it works for me if I grant full administrative S3 permission but not the same for single S3 bucket. Maybe someone made some more tests on that? I would like to avoid granting access to all S3 buckets to Veeam cloud account. I'm also quite suprised miminal permissions were not defined before this feature went life some time ago.

Dariusz

[wishr]

Hi Darius,

Thanks for bringing this to the table.

We have not tested this scenario specifically, so as of now it's recommended to use an account with full administrative permissions (the same is stated in our Cloud Credentials Manager guide section). We'll check internally if it's technically possible to narrow down the permissions to a single S3 bucket.

Regards,
Fedor

[veremin]

The said document is in works. You can give a shot to minimal IAM policy created for Direct Restore to EC2; might work for Capacity Tier as well. Thanks!

[dariusz.tyka]

Hi Eremin,

unfortunately event this minimal set of permissions include:
s3:CreateBucket",
"s3:ListBucket",
"s3:ListAllMyBuckets",
"s3:DeleteBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetBucketLocation",
"s3:PutLifeCycleConfiguration",
"s3:GetObject",
"s3:RestoreObject",
"s3:AbortMultiPartUpload",
"s3:ListBucketMultiPartUploads",
"s3:ListMultipartUploadParts

with following permission:

"Effect": "Allow",
"Resource": "*"

what in fact grant access to all S3 buckets.

[veremin]

And it wouldn't work if you try to limit resource parameter to the given bucket?

Code: Select all

"Resource": "arn:aws:s3:::my_archivetier_bucket/*"

Thanks!

[dariusz.tyka]

Hi Eremin,

unfortunately not. I tried this way, also tried to grant full access to single S3 bucket but then during object storage creation I receive 'Invalid credentials for Amazon S3 endpoint'.

Dariusz

[dariusz.tyka]

Finally I got it working with limited set of permissions. I used this AWS link: https://aws.amazon.com/blogs/security/i ... -policies/
The policy looks like this - see below. Don't know if those are minimal permissions but anyhow those are much more strict than full access to all S3 buckets.

Code: Select all

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TheseActionsSupportBucketResourceType",
            "Effect": "Allow",
            "Action": [
                "s3:*" ],
            "Resource": [
                "arn:aws:s3:::Veeam_bucket_name" ]
        },
        {
            "Sid": "TheseActionsRequireAllResources",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:ListMultipartUploadParts" ],
            "Resource": [
                "*" ]
        },
        {
            "Sid": "TheseActionsRequireSupportsObjectResourceType",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject" ],
            "Resource": [
                "arn:aws:s3:::Veeam_bucket_name/*" ]
        }
    ]
}

[veremin]

Thanks, Dariusz, for sharing this valuable information. I will update this thread, once the official documentation is ready.

[dariusz.tyka]

Unfortunately the set of permissions I mentioned earlier is not sufficient. While it was OK and I could add Amazon S3 repository using this set of permissions the SOBR repository offload job fails with an error:

Processing 'Job name' Error: Amazon REST error: 'S3 error: Access Denied Code: AccessDenied', error code: 403

So for now I've changed the policy for AWS account used by Veeam to full S3 access. Awaiting for final set of permissions from Veeam.

[Tomsyr]

From Gostev's excellent weekly email, it is stated 'Update 4a is on track to be released this month'.
Will it include the Backup Copy functionality for S3?
Thx!
Tom

[Gostev]

No. Actually, in the same excellent weekly email, I did note that Update 4a is purely a bug fix update

[krzychu3000]

Hello,
I'm won't start a new thread.

We are using Veeam Backup & Replication in version 9.5.4.2753. I just added New IBM Cloud S3 Storege based on totorial: https://helpcenter.veeam.com/docs/backu ... l?ver=95u4 I can see this repository on Backup Infrastructure --> Backup repositories. But when I want to use this repository in Copy Job, I can't see newly added repository.

Restart of Veeam Application and whole veeam server didn't help.

[veremin]

You should add it as capacity extent to existing Scale-Out Backup Repository. Thanks!

[jsutton]

Is there any update on this whitepaper and or a minimum required permissions list for s3 compatible providers? Ive been looking for this for some time with no success and thought today i should go to the forums but looks like no dice here either.

[veremin]

You can give a shot to this set of permissions or to its modified version. Thanks!

[Shinji]

>Just as a heads up, there are a few of us internally working on a Cloud Tier Deep Dive White Paper which will contain explanations around >scenarios like this. We hope to have it out in 4-6 weeks.

Is this white paper published already?

[wishr]

Hi,

Not yet, but it's coming!

Thanks

[Shinji]

I really want to have it.

Thanks in advance,