Using object storage as a backup target
Post Reply
pirx
Expert
Posts: 129
Liked: 13 times
Joined: Dec 20, 2015 6:24 pm
Contact:

how to cleanup Veeam S3 buckets?

Post by pirx »

Hi,

how can we cleanup potential leftovers in S3 buckets? It seems that due to DB issues we not only have (very) old unused vbk/vib etc files on our local repositories, there might be some old files in S3 too. With local repositories it's quite easy to compare what is on disk with what the DB knows. I've not much experience with S3 but when I browse a S3 bucket I don't find the vbk/vib files I see on local storage. Thus it's hard to find something in DB. Browsing a S3 bucket I see vbm and AUX-Data "files" in objs/.

Maybe it's just my lack of knowledge, but how can I get the actual filenames that are stored in S3 object storage?

pirx
Expert
Posts: 129
Liked: 13 times
Joined: Dec 20, 2015 6:24 pm
Contact:

Re: how to cleanup Veeam S3 buckets?

Post by pirx »

I played a bit with aws cli.

Code: Select all

aws s3api list-objects --bucket "veeam-backup-fra" --prefix "Veeam/Archive/xxxx/xxxxxx/xxxxxx/objs"
I get the objects but they clearly do not match any vbk/vib. I understand that data is stored differently in S3 than on NAS or other storage. From the timestamps I get the feeling that we really might have some old data, as Veeam support suspects. Still, there seems no script or anything from Veeam that can perform a manual cleanup. In theory everything should be consistent and the various default Veeam cleanup task should take care of it. But as we found large amounts of old data on local SOBR extents, I guess this could be the same on S3 (but probably more expensive over time).

veremin
Product Manager
Posts: 18009
Liked: 1718 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: how to cleanup Veeam S3 buckets?

Post by veremin »

You can check how the data is stored in object storage here, but we do not recommend cleaning up anything manually. Open a support ticket and let support engineer identify the reason why the files were left over on object storage and have been removed since then.

If you have an open ticket already, provide its number here, so that, we can follow the investigation.

Thanks!

Gostev
SVP, Product Management
Posts: 27414
Liked: 4544 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: how to cleanup Veeam S3 buckets?

Post by Gostev »

pirx wrote: Nov 26, 2020 9:39 amhow can I get the actual filenames that are stored in S3 object storage?
Unlike block or file storage, object storage has no file system - so there are no files to start with... it's just a bunch of objects, each belonging to one or more restore points.

pirx
Expert
Posts: 129
Liked: 13 times
Joined: Dec 20, 2015 6:24 pm
Contact:

Re: how to cleanup Veeam S3 buckets?

Post by pirx »

I understand this. But compared to storage with a filesystem we completely have to rely on Veeam that everything is cleaned up.

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests