Guide for Backblaze SOBR?

[globalpartsIT]

I apologize in advance but I don't have 800 hours to read through this forum to find what I am looking for. I've just spent nearly 4 hours reading everything with "backblaze" in the thread. I've also read the entire Veeam B&R v10 User Guide and still can't answer my own questions.

Is there a simple, easy to read tutorial for how to use Backblaze to store immutable backups in the cloud, something that doesn't require you to be a certified Veeam engineer to understand? Like "click here", "enter this", etc.? Something that doesn't want you to start your entire backup set over from scratch, destroying weeks/months worth of backups?

I followed the Backblaze guide to a T but I'm seeing a ton of errors in Veeam B&R that it is unable to delete objects in the SOBR. I'm guessing that I set the immutability number of days incorrectly in Backblaze but I am not certain, it could be the setting in the backup jobs. Most of my jobs run once per day in the evening. One job, the backup of our SQL server, runs hourly. I just want these backups copied to Backblaze, to be immutable for a set number of days and to be protected from bad actors. I would like to be able to follow a Best Practices guide in this regard as I am not an expert in this area nor do I wish to become one.

[Gostev]

I'm seeing a ton of errors in Veeam B&R that it is unable to delete objects in the SOBR. I'm guessing that I set the immutability number of days incorrectly in Backblaze

Sounds like this is the problem. You should not be setting any immutability settings on the Backblaze side at all... Veeam will take care of managing immutability for you, if you enable the corresponding setting in the Object Storage Repository wizard.

[Andreas Neufert]

As from our side the settings are simple and documented for all vendors here:
https://helpcenter.veeam.com/docs/backu ... ility.html

Usually all the vendors have guides for how to set up things on their side with Veeam:
https://help.backblaze.com/hc/en-us/art ... Cloud-Tier

As Anton shared the bucket level immutability functionality you have enabled is not supported. Veeam need to set the immutability on object level to be able to update it with a higher value (date) as neeeded.

[globalpartsIT]

Thank you for the responses. My apologies for tardiness, I've been out on an emergency the last few days.

I checked my setup and my settings are correct; I, in fact, did not set any object lock timeframe in Backblaze B2, I was confused about that. The Backblaze guide that Andreas linked is the one that I used to get setup with the Backblaze SOBR. Same for the Object Storage Repository wizard, but I do have a question in regard to that: I did not begin with "brand new" backups, I simply modified current backups after creating the Backblaze SOBR such that future backup runs would also copy to Backblaze. Because I run backups every day I chose an immutability period of only 1 day. We forever incremental backups. Is 1 day a correct setting for this setup?

I have read the Block Generation section of the B&R v10 User Guide at https://helpcenter.veeam.com/docs/backu ... ml?ver=100 several times now. I can't wrap my head around how to make this work for my need, such that every day backups run successfully (they do), offload to Backblaze (they do) and then delete older backup files in Backblaze just as they do for my LAN storage (they don't appear to be doing this). The error message I'm seeing is:

DeleteMultipleObjects request failed to delete object [Veeam/Archive/Veeam_Backups/0bc9321e-9640-4a69-a018-1861a6dd80d5/00000000-0000-0000-0000-000000000000/blocks/3aa19271000c72e88164ab3dc79a68ce/194135.7dd4eb4b12334bfb131a84d7d884064c.00000000000000000000000000000000.blk] and [23] others, error: NoSuchVersion, message: 'Invalid version id specified'

I have no idea what "Invalid version id specified" means.

Should I just open a support ticket and have a tech assist me with getting Backblaze SOBR working correctly for my need? I hate to bother Support with something that I should be doing myself.

[Gostev]

1 day is not a common setting for immutability, as this will usually not be enough to detect an attack on your environment. However, it will not cause any issues or errors if you use it.

Sounds like you have everything set up correctly, and should just troubleshoot that deletion error with our support. Perhaps something is messing with the bucket's content, if our code can't find objects it put there. This is quite worrisome situation in any case, as normally Veeam must be the only software managing the content of the bucket, so objects can never just disappear from it.

[globalpartsIT]

Thank you, I will work with Support on this one.

[Andreas Neufert]

By any chance did you have enabled immutability on the bucket itself?

[globalpartsIT]

Not that I know of, I just followed the steps in the article to create the bucket and enabled Object Lock. I changed the immutability setting on the SOBR to 14 days instead of 1 and am going to wait to see how that affects the jobs. I don't think this is causing any issues other than taking up more storage in Backblaze than is needed but even that's not a big deal, storage is cheap. I have done a couple of file-level restores on one of the servers being backed up so it appears all Veeam functionality is still working fine. Other than the deletions of old files at Backblaze, that is.

[globalpartsIT]

Also, on a side note ... I am not receiving notifications from the forums. I checked Office 365 to see if they are being blocked and I do not even see any attempts to email me notifications. I checked my forum profile and it shows that I'm setup to receive email notifications. I'm subscribed to this topic and have it bookmarked as well. Not sure what's going on in this regard.

** UPDATE **

Scratch that. There was a checkmark next to "SUBSCRIBE TOPIC" so I thought I was subscribed. I clicked that, the checkmark disappeared and the message changed to "UNSUBSCRIBE TOPIC". So now I am officially subscribed. That behavior is the opposite of what I expected, I figured that simply posting to the topic would subscribe me to the topic. My bad.

[selva]

@globalpartsIT
Did you get this sorted out with support? I'm currently facing the same error (DeleteMultipleObjects failed with NoSuchVersion, invalid version id) and working with support. In my case this is fatal as it stops SOBR offload and my backups for the affected job have stopped getting offloaded.
object-storage-f52/persistent-offload-e ... 71154.html

Similar to yours, my bucket is exclusively used by B&R and the only setting in backblaze is object-lock enabled during creation.

[globalpartsIT]

@selva
Have not found a solution yet, have been waiting for Support to "look at the logs" but it seems to be taking a while. Maybe due to the holiday.

[Andreas Neufert]

Maybe it is the same situation as here: https://github.com/minio/minio/pull/11091
I will discuss with Backblaze...

[globalpartsIT]

Thank you, Andreas. I can't really answer that since I'm not a programmer, just a business manager. I just know that our Backblaze offloads are all failing now due to the "Invalid version id" error message. I have a support ticket open but have not received a response since the initial "can you upload your logs" response on 12/22. I uploaded the logs when I created the ticket on the support site and then uploaded them again in response to that message from Support. It seems that every time I create a ticket and upload logs as part of the process Support never knows about the logs I uploaded. Or the logs aren't actually uploading even though the site says they are. Whichever is the case I would like to get this issue resolved as soon as possible so that our Backblaze backups are working. The case number is 04559198. Thank you for your help.

[Andreas Neufert]

You can upload the logs as well on a plattform of your choice and share with out support. There is a "Contact a manager" button within support system that allows you to escalate the case.