Using object storage as a backup target
Post Reply
obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Usage of Job in Capacity tier

Post by obroni »

Is there anyway to find out the object usage (in GB/TB) of a single job in the scenario where multiple jobs are targeting a single SOBR? I'm thinking of the figure of actual usage after the deduplication due to hashing.

Gostev
SVP, Product Management
Posts: 29073
Liked: 5346 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Usage of Job in Capacity tier

Post by Gostev »

@Shestakov does Veeam ONE have such a report?

Shestakov
Expert
Posts: 7328
Liked: 776 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Usage of Job in Capacity tier

Post by Shestakov »

There are reports, such as Job History, showing how much data each job transfers to scale-out repository.
However, none of them shows how much backups data created by those jobs is moved to the object storage later on.
Could you clarify your goal, please? Do you need to know size of backups on the object storage or you want to know amount of transferred data?

dalbertson
Veeam Software
Posts: 468
Liked: 157 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Usage of Job in Capacity tier

Post by dalbertson »

@Shestakov A couple of scenarios pop into my head on this request.

1. VCSP - reporting tenant usage across a SOBR with multiple tenant jobs.
2. Enterprise - reporting usage from jobs linked to departments/divisions. - I know that they could setup individual SOBRs and then report a bit differently but most would probably be setup in a use case where multiple jobs are going to a single sobr.

@obroni Can you confirm what your specific use case is?
Dustin Albertson | Manager - Cloud & Applications | Veeam Product Management, Alliances

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

#1 is exactly my use case.

Want to understand how much capacity a customer is using in terms of archive storage.

Gostev
SVP, Product Management
Posts: 29073
Liked: 5346 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Usage of Job in Capacity tier

Post by Gostev »

In that case @Vitaliy S. also needs to be summoned here as VAC PM.

Vitaliy S.
Product Manager
Posts: 25263
Liked: 2211 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Usage of Job in Capacity tier

Post by Vitaliy S. »

This is not possible today (planned for next updates of VAC), however, @chris.arceneaux has created a script that allows reporting on capacity tier usage per tenant.

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

Just to clarify this is for a normal Veeam B&R installation being used in a multi tenant IaaS scenario, so in my head, not sure VAC would be where I would expect to see it.

I was more thinking it would be shown when right clicking on the backup under Backups->Object Storage. The current reported size seems to be based on the total logical size of the full backup files stored in S3, where I want to understand how much actual space is being used.

Vitaliy S.
Product Manager
Posts: 25263
Liked: 2211 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Usage of Job in Capacity tier

Post by Vitaliy S. »

Ok, then it seems like we need to have this reporting in Veeam ONE (for the IaaS scenario).

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

Vitaliy that sounds more like what I was expecting. Or if this information could be generated via powershell that would probably also be fine for my requirements.

Shestakov
Expert
Posts: 7328
Liked: 776 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Usage of Job in Capacity tier

Post by Shestakov »

Noted for future updates of Veeam ONE. Thanks

chris.arceneaux
Veeam Software
Posts: 448
Liked: 249 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Kentucky, USA
Contact:

Re: Usage of Job in Capacity tier

Post by chris.arceneaux » 1 person likes this post

@obroni As Vitaliy mentioned, I currently have two scripts for reporting Capacity Tier usage.
  • VBR - Per Jobs usage numbers (what you're looking for)
  • VCC - Per Tenant usage numbers
I'll DM you a link to the code shortly so you can test it out.

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

Thanks @chris.arceneaux . Did you send the scripts? I couldn't see anything when I looked.

chris.arceneaux
Veeam Software
Posts: 448
Liked: 249 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Kentucky, USA
Contact:

Re: Usage of Job in Capacity tier

Post by chris.arceneaux »

No problem. Please check your private message again.

dalbertson
Veeam Software
Posts: 468
Liked: 157 times
Joined: Jul 21, 2015 12:38 pm
Full Name: Dustin Albertson
Contact:

Re: Usage of Job in Capacity tier

Post by dalbertson »

@chris.arceneaux Why dont you put them in the veeamhub repo?
Dustin Albertson | Manager - Cloud & Applications | Veeam Product Management, Alliances

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

Thanks.

I've run the script but not convinced the numbers are correct? The 3 jobs that copy to object storage are showing as:
UsedObjectGB : 78246.91
UsedObjectGB : 69476.83
UsedObjectGB : 45113

But total object usage on repository is: 41.7TB

Are these pre-compression or something like that, or am I confusing the numbers?

chris.arceneaux
Veeam Software
Posts: 448
Liked: 249 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Kentucky, USA
Contact:

Re: Usage of Job in Capacity tier

Post by chris.arceneaux »

Values are being pulled from Backup Job metadata. Please check the numbers shown in your console (see screenshots) to see how they match up and let us know.

Console view:

Image

Script output:

Image

Here's another POV showing actual usage from the object storage itself:

Image

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

Ah ok, that might explain it then. That's what originally prompted this thread, that object storage total for each job, seems equal to the logical size of all the compressed full backups, rather than after object deduplication and thus the actual underlying storage being consumed?

chris.arceneaux
Veeam Software
Posts: 448
Liked: 249 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Kentucky, USA
Contact:

Re: Usage of Job in Capacity tier

Post by chris.arceneaux »

If that is the case, can you please open a support case on this for visibility? The backup job metadata should have the correct numbers to be used for usage reporting.

Please post the case number as well so I can assist.

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni » 1 person likes this post

Case ref 04057065 raised

AlexHeylin
Veeam Legend
Posts: 123
Liked: 23 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: Usage of Job in Capacity tier

Post by AlexHeylin »

Did this get resolved so the script can report the actual usage of the S3, and if so please will you consider adding the scripts to Github @chris.arceneaux?
Thanks

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

As far as I can tell this is still an issue

Vitaliy S.
Product Manager
Posts: 25263
Liked: 2211 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Usage of Job in Capacity tier

Post by Vitaliy S. »

There were no changes in how Veeam B&R calculates usage since there were no major releases yet. I've asked our QA team to review this in v11. Once I have more information, I will update the thread.

Ci2Group
Service Provider
Posts: 8
Liked: 1 time
Joined: Dec 20, 2016 3:16 pm
Contact:

Re: Usage of Job in Capacity tier

Post by Ci2Group »

Was this script added to the veeam hub or is it possible to get it via different means ?

chris.arceneaux
Veeam Software
Posts: 448
Liked: 249 times
Joined: Jun 24, 2019 1:39 pm
Full Name: Chris Arceneaux
Location: Kentucky, USA
Contact:

Re: Usage of Job in Capacity tier

Post by chris.arceneaux » 1 person likes this post

The script in question was never released publicly as it used undocumented API calls that are no longer valid using the latest version of our product.

Starting with Veeam Service Provider Console v5, Performance/Capacity/Archive Tier usage is now accessible via:

obroni
Service Provider
Posts: 125
Liked: 21 times
Joined: Nov 21, 2014 10:50 pm
Full Name: Nick Fisk
Contact:

Re: Usage of Job in Capacity tier

Post by obroni »

I believe the new Service Provider Console handles scenario #2 which was highlighted near the top of the thread.

Has any progress been made towards scenario #1, calculating usage of a "tenant" in a shared SOBR repository in VB&R, as you would find in a MSP's IaaS platform?

Vitaliy S.
Product Manager
Posts: 25263
Liked: 2211 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Usage of Job in Capacity tier

Post by Vitaliy S. »

Actually, the Service Provider Console resolves the 1st case (reporting tenant usage across a SOBR with multiple tenant jobs), but not yet the 2nd where SOBR is used remotely/on-premises and is not part of the VCC infrastructure.

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests