Discussions related to using object storage as a backup target.
Post Reply
DWZH
Novice
Posts: 3
Liked: 2 times
Joined: Sep 18, 2015 9:22 am
Contact:

Feature request - better overview and statistics about S3 background jobs and buckets (offloading, synchronizing)

Post by DWZH »

Hi Anton

I really like the object storage integration. We tested it starting from the pre-Alpha version as a partner. From day 1 we had Projects with ca. 500TB (one big Enterprise Customer) and some smaller ones (30-100TB per customer). Unfortunately since then the overview about background task is still the same and not really help full. Therefore I would like to have a better overview and statistics.

Example1: ReSync of the repository
Sometimes Veeam has to resync the repo and therefore it will read all the metadata from the bucket. This operation can last long and has no idication of the progress. Only how long it is running. It would be handy to have a status like xTB of y TB resynct, a % indication with an guess about ETA and something like average rescanned objects per seconds. With this it would it make much easier to troubleshoot for bottlenecks.

Example2: The offload task overview is something that gives you some information that it is running, but not much if you need to know if really all backups where offloaded. A lot of times on smaller VBR installation I see on the running processes "backupjobname_Offload" as running but 0%. On this task I just see "pending". The question would be: why is it pending? What resource is needed?
If the backup repository is not ready there is a message "waiting for backup repository resources". Here I would like to see more info to make the operations of the daily business and the troubleshooting easier.

Another point that in my opinion needs more infos and background statistics are the buckets. Veeam puts in all the objects and has a local index. Therefore all information are on the VBR. How about making this info available with an statistic overview
  • How many object with which size are in each buckets. Useful to see the object size if you need to troubleshoot things like the storage optimization for the backup jobs with multiple jobs and long retention (like backup copy jobs)
  • A short list with all buckets, the used size, object count per size, quota, object lock yes / now, S3 endpoint - This short list would be helpful for troubleshooting and for documentation
Thank you very much
Gostev
Chief Product Officer
Posts: 31349
Liked: 6599 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature request - better overview and statistics about S3 background jobs and buckets (offloading, synchronizing)

Post by Gostev »

Hello!

Thanks for your kind words about our object storage integration.

1. Agree, @veremin please discuss with developers if this is possible.

2. Already today, the offload session should always have the reason why it is pending. Do you mean we should add the repository name for which we're waiting for a task slot before we could start offload?

3. It does not seem useful to see some arbitrary number of objects like 12789234 objects. What are you going to do with this information? Also, the size of each object will be different, while average object size is about the same in any environment: approx. 2x less than backup file's block size.

4. Most of this information should be available already in the daily SOBR email report. You say "all buckets" though - does it mean you have multiple SOBRs configured, and want the consolidated report? I wonder if such report is already coming Veeam ONE v11 @wishr?

Thanks!
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Feature request - better overview and statistics about S3 background jobs and buckets (offloading, synchronizing)

Post by wishr »

Hi guys,

We have an FR for a specific report for object storage in the VONE backlog for a while and will see if we could get back to it after v11. Btw, thanks for sharing the details on what data would be interesting for you.

At the same time, some of this data is already available in VONE UI and SOBR configuration report, and more enhancements are coming in v11!
veremin
Product Manager
Posts: 20258
Liked: 2248 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Feature request - better overview and statistics about S3 background jobs and buckets (offloading, synchronizing)

Post by veremin »

1. Makes sense, indeed. We will investigate this internally - I will update the topic with the results of our findings. Thanks.
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests