S3-compatible object storage

[alexchern]

Hello.
The question is related to adding S3-compatible object storage. Company-developed proprietary BLOB stores in ceph have their own implementation of the S3 surface. When adding S3, you can select a bucket, but when you select folders inside the bucket or when creating a new STORAGEClass error, the storage class uses erasure code.
The question is what storageclass does veeam12 use?! since it worked on version veeam 10.
How can I see through the logs which storageclass is trying to connect to the storage?

[Andreas Neufert]

We reduced the supported storage classes because of issues that we faced in Support. At this time we support only "STANDARD" storage class.

[sfirmes]

Andreas is correct that VBR only uses the storage class named "STANDARD" when we write backup objects to object storage. As he noted, this change was part of VBR v12 due to many issues in the field with the usage of non-standard storage class names. I wrote a blog last year OSNEXUS and Auto-Tiering which explained how they are able to use storage classes other "STANDARD" while maintaining compatibility with our software. I mention this because their object storage is Ceph based.

We do use other storage class names that are "valid" AWS S3 storage class names for some special purposes, for example VBR's Archive tier where we use "GLACIER" for example. Here is the list of the "valid" AWS S3 storage class names AWS S3 Storage Classes. VB365 also supports several of these storage classes and you can find them here VB365 Supported Amazon S3 Storage Classes

Hope this helps.

[alexchern]

Hi. I understand correctly that through the OSnexus program you can configure S3 in veeam if we use other storage classes.

[alexchern]

Or rather, as I understand the integrated approach, data storage system + OSNexus

[sfirmes]

Hi. I understand correctly that through the OSnexus program you can configure S3 in veeam if we use other storage classes.

I used OSNexus and their QuantaStor product as an example of how it is possible to use Ceph to be a target for GET/PUT api calls using the STANDARD storage class, but in the background Ceph is storing the objects using a storage class other than STANDARD. I have worked with the OSNexus folks and they have a tremendous understanding of how to make Ceph work well with our software. Check out their Veeam page and hopefully you find the information there helpful.

[alexchern]

thank you for your answer

[osnexus]

@alexchern for more info on it there's a video here on how to configure the auto-tiering/intelligent data placement in QuantaStor here (https://youtu.be/TDlRqehpMUs?feature=shared&t=1126). As @sfirmes notes we intercept the S3 PUTs and redirect the objects to different object storage tiers based on user set rules/criteria. That video here goes into the rules configuration at 18:46 min/sec in, the rest of the video is all on how to setup and object storage cluster from scratch. QuantaStor has embeded LUA code in the Ceph RGW to dynamically route the objects to the right Storage Class based on your rules. As you've probably seen in your testing it is important to route the small objects to replica=3 flash storage (and larger objects to EC storage). Note also that it is ideal to have the STANDARD storage class as all-flash replica=3 as there's a small stub that goes into the STANDARD storage class data pool even if the data objects are routed to another storage class. If you're interested in giving it a go there's a free Community Edition available on our web site (320TB cluster size max for CE) available here (www.osnexus.com/downloads) and if you'd like some assistance getting everything setup just grab Trial Edition licenses and someone will reach out.