Visibility of (Orphaned) Files in Object Storage

[ratkinsonuk]

Good afternoon all. I'm getting used to Veeam's 'real world' functionality around object storage, but have a question about visibility of files/backup data.

I accidentally sent one of our backups to the wrong AWS S3 repository (REPOSITORY_X). I'm not too concerned with that as I'm testing with non-critical systems. I moved the backup to the correct repository (REPOSITORY_Y), and can see data being sent to it.

However, the 2TB of data I accidentally sent to REPOSITORY_X seems to be completely invisible within Veeam, even though I have other backups and repositories pointing the that location.

I've done re-scans, and that doesn't make any different. If this was a file-based repository, I'd expect to see the data in a number of places, even though it's effectively orphaned.

Is this what I should expect? Veeam understands the location and directory structure, so I don't get why it's not importing the VBK files as I'd normally expect.

Cheers, Rob.

[Gostev]

Object storage does not have a file system, so there's no concept of a "file" or a "folder" to start with... these simply do not exist on object storage. There's not even a "directory structure": just a bunch of objects in a bucket, and particular UI client's choices on how to group and display them to you.

Backup files is a thing that only exists in a world of block-based and file-based (NAS) backup repositories. If you happen to accidentally copy an existing backup file to object storage, it just creates a single huge blob (object) that our product has no idea what to do with because we do not store backups on object storage like that (as a single blob containing "normal" backup file).

[ratkinsonuk]

Thanks for the response Gostev. Hope life's getting better for you by the way, you're weekly additions are very much missed!

From the AWS GUI, the data has both a folder and file structure. In fact, when I created the Veeam repository, I had to give it a folder name to create.

Is that not visible to Veeam through the S3 API's? Is it worth me logging an enhancement request to develop similar functionality to file storage, so that we can import, view and delete S3 data in the same way?

If none of this is possible, how does the community manage their S3 data? Veeam often gets 'disconnected' from the backup files, but at I can manage the reporting and deletion of the data from the Veeam GUI. Are we saying I've got to do this from AWS going forward?

Thanks, Rob.

[Gostev]

The AWS S3 bucket explorer GUI simply chooses to display objects with the same prefix in their name as residing in a directory... this "directory" is a virtual UI-only object that does not exist in reality.

Veeam is already able to import, view and delete S3 data that it creates in object storage.

You should absolutely never manage Veeam data in object storage outside Veeam with a single exception: when you want to delete everything Veeam created. At least not without an explicit guidance and hand holding from our Technical Support. Otherwise, the results will be completely unpredictable.