Using object storage as a backup target
mdiver
Service Provider
Posts: 87
Liked: 15 times
Joined: Nov 04, 2009 2:08 pm
Location: Heidelberg, Germany
Contact:

Immutability with on-prem S3 storage

Post by mdiver »

As far as I get it, minio should support the S3 object lock feature:
https://docs.min.io/docs/minio-client-c ... guide#lock

Could that be leveraged by the immutability function within 9.5 U4?

Regards,
Mike

wishr
Veeam Software
Posts: 1865
Liked: 220 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Immutability with Minio possible?

Post by wishr »

Hi Mdiver,

v9.5U4 doesn't support immutability at all, it's a v10 feature.

In general, v10 supports immutability feature on any S3-compatible devices which implements S3 Object Lock API.

However, Minio did not release this API until most recently, so we have not been using it in our QC labs for v10 to validate their API implementation. I have spoken to our QC leadership about it, and they will schedule this testing for post v10.

Thanks

edirschedl
Enthusiast
Posts: 38
Liked: 4 times
Joined: Jul 21, 2016 12:29 pm
Full Name: Emanuel Dirschedl
Contact:

Re: Immutability with Minio possible?

Post by edirschedl »

@wishr: Do you have a list of tested object storage devices with the V10 immutability feature?

Gostev
SVP, Product Management
Posts: 26887
Liked: 4360 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutability with Minio possible?

Post by Gostev »

We've been testing with Cloudian, since there were not too many alternatives at the time.

edirschedl
Enthusiast
Posts: 38
Liked: 4 times
Joined: Jul 21, 2016 12:29 pm
Full Name: Emanuel Dirschedl
Contact:

Re: Immutability with Minio possible?

Post by edirschedl »

Do you plan to integrate the immutability feature as criteria for VEEAM Ready, after releasing V10?

DGrinev
Expert
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: Immutability with Minio possible?

Post by DGrinev »

I have heard it will be a part of the Veeam Ready program.

Thanks!

Gostev
SVP, Product Management
Posts: 26887
Liked: 4360 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutability with Minio possible?

Post by Gostev »

But optional part, of course.

edirschedl
Enthusiast
Posts: 38
Liked: 4 times
Joined: Jul 21, 2016 12:29 pm
Full Name: Emanuel Dirschedl
Contact:

Re: Immutability with Minio possible?

Post by edirschedl »

Does someone has a list of object storage products where S3 object lock is already possible? I've the feeling that there is not much choice in this area currently.

I know only from AWS (of course), Cloudian and Minio currently after reading this thread.

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with Minio possible?

Post by sfirmes » 3 people like this post

We are currently working with our technical alliance partners who support the S3 object lock feature as they continue to test their offerings with our V10 immutability feature. We are also working on including optional immutability tests in our Veeam Ready Object program for v10 for any alliance partner who supports S3 object locking.

Gostev
SVP, Product Management
Posts: 26887
Liked: 4360 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutability with Minio possible?

Post by Gostev »

edirschedl wrote: Feb 04, 2020 1:18 pmI know only from AWS (of course), Cloudian and Minio currently after reading this thread.
You can also add Zadara to that list.

mengl
Service Provider
Posts: 9
Liked: 8 times
Joined: Oct 19, 2018 7:02 am
Full Name: Michael Engl
Contact:

Re: Immutability with Minio possible?

Post by mengl »

Did a quick test today with minio.
Failed to enable backup immutability: the selected object storage does not support S3 Object Lock feature
Does this have to be enabled in minio?


Minio DEVELOPMENT.GOGET most current of today
VBR v10 RTM 10.0.0.442

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with Minio possible?

Post by sfirmes » 6 people like this post

Yes they do support the S3 object locking. This isnt their WORM setting. That got me on my 1st tests.

I am testing minio running on Centos 8 with our v10 RTM code using the S3 API CLI.

I used the following commands to create the locked/immutable bucket:

Code: Select all

[root@Minio-Centos minio]# mc mb myminio/testlock --with-lock
Bucket created successfully `myminio/testlock`.

[root@Minio-Centos minio]# mc lock myminio/testlock compliance 30d
COMPLIANCE mode is enabled for 30d

This is how I was able to confirm it.

[root@Minio-Centos minio]# aws --no-verify-ssl --endpoint-url https://minio:9000 s3api get-object-lock-configuration --bucket testlock
{
    "ObjectLockConfiguration": {
        "ObjectLockEnabled": "Enabled",
        "Rule": {
            "DefaultRetention": {
                "Mode": "COMPLIANCE",
                "Days": 30
            }
        }
    }
}
During the next few weeks, I will be running minio through other testing via the v10 gui and will update this thread with any tips or issues that may help you.

mengl
Service Provider
Posts: 9
Liked: 8 times
Joined: Oct 19, 2018 7:02 am
Full Name: Michael Engl
Contact:

Re: Immutability with Minio possible?

Post by mengl » 2 people like this post

Thanks. Thats it!
With the lock option configured I was able to add it with immutability enabled.

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with Minio possible?

Post by sfirmes » 1 person likes this post

sfirmes wrote: Feb 05, 2020 9:14 pm Thanks. Thats it!
With the lock option configured I was able to add it with immutability enabled
Were you able to verify if the immutability was working? In my testing I got errors during the SOBR offload process due to versioning not being on the Minio bucket. Image

We require versioning to be enabled on the bucket when object locking is enabled. Minio doesn't support versioning and based on their feature request page they don't intend to https://github.com/minio/minio/pull/649 ... -497511953

You can verify if you have versioning enabled by the using AWS S3 API cmd "get-bucket-versioning" https://docs.aws.amazon.com/cli/latest/ ... oning.html

The syntax to use the AWS S3 API with the Minio cli is here:
# aws --no-verify-ssl --endpoint-url https://<>ip address or server name>:9000 s3api get-bucket-versioning --bucket <bucket name>

I will continue to work with Minio and will update this thread if I learn anything regarding getting Minio to work with our immutability feature.

y4m4
Technology Partner
Posts: 1
Liked: 3 times
Joined: Feb 10, 2020 8:03 pm
Full Name: FNU Harshavardhana
Contact:

Re: Immutability with Minio possible?

Post by y4m4 » 3 people like this post

MinIO doesn't support versioning yet but it is being actively worked on https://github.com/minio/minio/tree/xl-v2

The relevant issue to track https://github.com/minio/minio/issues/2 ... -577406100

Gostev
SVP, Product Management
Posts: 26887
Liked: 4360 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutability with Minio possible?

Post by Gostev »

Thanks, Fnu. I see you're from MinIO, so I marked your forum account as TAP. Please, keep the community posted on the progress.

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with Minio possible?

Post by sfirmes »

When the versioning is supported by Minio, I will do some testing and post the results in this thread.

poulpreben
Veeam Vanguard
Posts: 1008
Liked: 432 times
Joined: Jul 23, 2012 8:16 am
Full Name: Preben Berg
Contact:

Re: Immutability with Minio possible?

Post by poulpreben » 2 people like this post

Gostev wrote: Feb 04, 2020 4:11 pm You can also add Zadara to that list.
You can add Ceph version >= 14.2.6 to that list.

dalbertson
Veeam Software
Posts: 292
Liked: 74 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutability with Minio possible?

Post by dalbertson »

Ceph would need to be tested with Veeam Ready. @poulpreben

@sfirmes Lets discuss this today
Dustin Albertson | Manager - Cloud & Applications | Veeam Product Management, Alliances

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with Minio possible?

Post by sfirmes »

Thanks @poulpreben and @dalbertson. When any Ceph testing is done, I will post the results.

Gostev
SVP, Product Management
Posts: 26887
Liked: 4360 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Immutability with on-prem S3 storage

Post by Gostev » 2 people like this post

Just wanted to update that both Cloudian and Zadara have passed Veeam Ready for Object Storage, including optional immutability support verification. Thanks!

UniMatrix
Novice
Posts: 6
Liked: never
Joined: Dec 04, 2015 9:18 am
Full Name: Erik van der Neut
Contact:

Re: Immutability with on-prem S3 storage

Post by UniMatrix »

Started following this topic as the idea is taking shape to add an expansion cabinet to our Synology RS3617RPxs (Backup Copy target in another building) and run Minio on Docker on it to present an immutable datastore to VeeAm.

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with on-prem S3 storage

Post by sfirmes »

Currently Minio isn’t compatible with our immutability feature due to them not supporting versioning. I expect them to have versioning support soon and when they do they will be tested. I’ll update this thread when they successfully pass the tests.

Andreas Neufert
VP, Product Management
Posts: 4598
Liked: 877 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: Immutability with on-prem S3 storage

Post by Andreas Neufert » 2 people like this post


Mattias
Influencer
Posts: 11
Liked: 2 times
Joined: Jan 21, 2015 1:18 pm
Contact:

Re: Immutability with on-prem S3 storage

Post by Mattias »

Hi !

Does immutable object lock work/supported on NetApp S3 GRID on prem ? Anyone know ?

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with on-prem S3 storage

Post by sfirmes » 1 person likes this post

@Mattias The NetApp S3 GRID doesn't have this feature yet. When it does we will update the list noted above.

HendersonD
Expert
Posts: 147
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

[MERGED] Immutable backup ransomware protection?

Post by HendersonD »

In the What's New for Veeam 10 is this

"The new immutable backups feature protects your recent backup copies from modification and deletions by leveraging advanced object storage functionality called S3 Object Lock. You can create immutable backup copies on Amazon S3, as well as any S3-compatible service provider or on-premises object storage solution which implements S3 Object Lock AP"

We are looking to take advantage is this feature. Are there any "on-premises object storage solution which implements S3 Object Lock AP"? We use Nimble Storage, does it have this feature? Do I need to contract with Amazon to get a target that uses this feature?

wishr
Veeam Software
Posts: 1865
Liked: 220 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Immutable backup ransomware protection?

Post by wishr »

Hi HendersonD,

I'm merging your question with an existing tread. Please, take a quick look and let us know if you have any additional questions.

Thanks

sfirmes
Veeam Software
Posts: 97
Liked: 56 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Immutability with on-prem S3 storage

Post by sfirmes »

@HendersonD There is a link in a previous post by Andreas in this thread with a link to the unofficial compatibility list.

dalbertson
Veeam Software
Posts: 292
Liked: 74 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Immutability with on-prem S3 storage

Post by dalbertson »

@HendersonD as stated above we will list the vendors who passed Veeam ready testing on the veeam ready page.

https://www.veeam.com/ready.html

object-storage-f52/unoffizial-compatibi ... 56956.html

Right now netapp isn’t on there. A few others are close. You could leverage S3
Dustin Albertson | Manager - Cloud & Applications | Veeam Product Management, Alliances

Post Reply

Who is online

Users browsing this forum: cthorpe and 9 guests