Storage used on SOBR per VM

[Kazz]

Hi,

Looking for some help calculating space used per VM on SOBR.

I have per-machine backup files enabled on my backup repository. If I go to backups -> disk and open properties of a back up job I can highlight any VM, look at the backup size column and adding all numbers up gives me a total size used by the backups for any VM selected. (Why can't veeam do the calculations for us?)

If I go to my backup repository file system and highlight all of the VIB and VBK files (to see the size used) for the same VM I just manually calculated backup size for I always come up with the same number.

This is easy to do when you have a handful of backups.

My problem is with backups in capacity tier. I have over 300 restore points for many VMs and no way of knowing how much space is used by back up for each VM unless I want to spend days manually adding backup size values for each vm. I hope I am wrong and there a way to tell how much space is used by backups in SOBR. The only value I see is total space used in object storage, but no break down per VM.

Is there way to get per VM backup space used in object storage?

Thank you

[chris.childerhose]

Does your SOBR use NTFS or ReFS, and is Capacity Tier enabled? That will determine a few things on how people answer including me.

[Kazz]

Hi Chris,

My SOBR uses ReFS and Capacity Tier is enabled.

Capacity tier is where I need to calculate the size of the back up for each VM, but don't see any practical way of doing so unless I add sizes of 300+ restore points manually.

Thanks

[chris.childerhose]

This thread on the forums may help as someone was looking for a similar thing - powershell-f26/v11-get-vbrbackupreposit ... 72415.html

[Kazz]

Chris,

I checked every single post in the thread you suggested, but most of the post have code that does not work. Is there a specific post you wanted me to check?

I am not finding any of them useful in my situation.

Thanks

[Kazz]

Sorry if I caused any confusion, and didn't mention that my capacity tier is Amazon S3 object storage and my performance tier is a local disk formatted as ReFS.

[chris.childerhose]

Hi @Kazz - you can try this site as there is a tool called "Blockstat" which is great for ReFS calculations you want to see - http://dewin.me/refs/

[Kazz]

Chris,

I don't need to calculate how much space is used on ReFS, I need to calculate the space utilization per VM on S3 object storage.
Each VM has more than 300 restore points

[chris.childerhose]

Apologies for misreading your ask. I will see if I can find out how this can be done but it will not be as easy as the performance tier.

[lowlander]

Hi Kazz,

good point. I also noticed that in some occasions the capacity tier is not reducing the storage, but instead overloading the storage in the capacity tier. This is I think related to the block size that can't be reduced by 50% on the capacity tier ( reduction ratio )

I found out it can be an option to determine the capacity of the capacity tier, just by measuring the buckets or folders within the buckets.

With the aws cli you can try to execute the following command to the object storage : aws s3 ls --endpoint http://[ENDPOINT(S3)] s3://[bucket]/ --recursive --human-readable --summarize > [bucket].csv

This creates a bucket.csv file where you see all objects with the corresponding block size for the objects ( I think this represents the reduction ratio ).

Block size matters regarding the following math:
- 4096 KB ( local target large blocks ) will result in an optimal blocksize of 2048 KB. 10 TB will result in about a first full backup of ~5Mio objects ( 5.252.880 ) and for a offload window of 2 hours it will need 1.4 Gbps ( 1456 MBps ) of throughput towards the capacity tier.
- 8192 KB ( local target extra large blocks ) will result in an optimal blocksize of 4096 KB. 10 TB will result in about a first full backup of ~2,5Mio objects ( 25.165.824 ) and for a offload window of 2 hours it will need 1.4 Gbps ( 1456 MBps ) of throughput towards the capacity tier.

Increasing the blocksize will result in less objects, however I think this also impacts the capacity on the object storage layer ( for example, incremental backups but also backup vms that will not benefit from a reduction ratio. Increasing the offload window on the SOBR will decrease the needed bandwith in this period of time.

Maybe off topic, however also a thing I don't can explain for 100%. When we have a virtual machine running on Storage array A, and it is being moved to Storage array B. When veeam backups this vm in the next cycle, I assume that the incremental will blowup the datasize of the virtual machine being backed up. Assuming that we do a copy out where already 5 monthly backups are archived, will every new monthly backup will consume this additional data size ?

Thing I also want to bring into this discussing is what would be preferable practices to follow when having on sobr with a capacity tier and multiple jobs pointing to this sobr.

Regarding the preferable practices :
- What should be the maximum bucket size;
- Use one job per SOBR ( per bucket ) or multiple jobs per SOBR ?
- When to use larger block sizes ( for example there is a reg key available for extra large block sizes (8096) ) ?
- When setting the offload windows on a sobr, will this automatically increase the calculated bandwidth above, or does it just take all bandwidth that is available ?
- How to deal with capacity needed for long term retention in the capacity tier when migrating virtual machines between storage arrays ?

SOBR's are great to use, but I think it should be valuable for VBR to give some more insights in how we can correlate backup data in the capacity tier with the corresponding backup data that belongs to virtual machines being backed up. I guess in the database of Veeam, a lot of information can be correlated.

In the VBR console you can also deep dive in the objectstorage folder in the backup context. What I noticed here is that when compression is 1.0 there will be an increase in object storage regarding to the backup data. As a feature request I would like to add a column besides Backup Size, that lists the backup on Object storage per virtual machine. This helps predicting the needed capacity on object storage, on more granular level ( instead of the total amount that is now visible )

[Kazz]

https://vnote42.net/2021/11/30/query-ba ... owershell/

I used "Analyze used Repository space" script posted in the blogpost above to get results exported to a CSV file. Then I got an average daily change size from my incremental backups and replaced all full back up sizes except for the 1st one with the average number I got. Adding all numbers together gave me a rough idea of how much space is being user per VM in SOBR.

This is done based on the assumption that my full backups taken once a week are not really uploaded to SOBR.