Monitoring for Scale-out repository to AWS

[asim.sinha]

how can I monitor the AWS mirroring (Scale out repository) of backup files ?
I have recently configured a scale-out repository to send a copy of backup files to AWS S3. I want to know how can we know or monitor if the backup files are getting copied to S3? Is there any way it can be monitored?

[wishr]

Hi Asim,

You can view this data in the Capacity Tier session statistics.

Also, you may generate some Veeam ONE reports showcasing information on backups stored on SOBR and its extents, for example the SOBR Configuration report.

Thanks

[Gostev]

The easiest is to just look through the daily SOBR status email report, this will indicate any mirroring errors.

[veremin]

This is the report Anton's talking about. Thanks!

[pirx]

How can I get a report about offloaded backups, something like the protected vms report for backup or copy jobs? We are currently facing the issue that not all backups are offloaded, no error or warning. So I need to check all of our VMs for their offload status. I did not find anything in Veaam One for this

[veremin]

Generally, SOBR daily report should be an indication of this type of issues. Thanks!

[pirx]

But not if ther is no error, right?

[pirx]

What I really need is a report of last version (or better history) of backups, offloads and copies for VMs. Currently I can only get this for backup and copy. Right now something is going wrong with offloads, so it would be good to immediately see how many VMs are affected. In general we must send reports from Veeam One to different parties and they ask for some time now if we can add VMs status on S3 capacity tier.

[wishr]

Hi Pirx,

This is something we are considering adding in the next VONE versions. Thank you for the feedback.

Could you please let us know how an "ideal" report should look like?

[pirx]

Not sure what the ideal report would be, but adding Offload to the list of "Job Types" in Veeam One would definitely be a start. Just like Backup, Backup Copy, Replication.

Any idea if there is some other way (powershell, SQL)to get this information? It's really messy here, currently I've several dozen offload tasks with hundreds of VMs that are not offloaded. Most of them without error, just no offload. So I need to get an overview.

[oleg.feoktistov]

There is an unofficial (unsupported) .NET method to query offload sessions and present them as regular backup sessions, which we do through powershell:

Code: Select all

$offload = [Veeam.Backup.Model.EDbJobType]::ArchiveBackup
$freezing = [Veeam.Backup.Model.EDbJobType]::ArchiveFreezing
$jobTypes += $offload, $freezing
$sessions = [Veeam.Backup.Core.CBackupSession]::GetByJobType($jobTypes)
$sessions 

There is no data on which VMs are being offloaded precisely, but it has pretty much everything concerning status, results and session progress. We also have plans to support offload sessions with official powershell cmdlets.
Thanks!

[pirx]

Thanks, in my case this does not help as there were no offload sessions at all for the affected VMs. I'd really need the information about offloaded backups for each VM. I hope there will be something in the future. In the end an offloaded backup is not different to me than a classic copy job.

We have a backup type field in our CMDB (Veeam, No, other backup tool...), this field is added as vSphere tag to every VM. Veeam One Protected VMs report is created every morning based on that tag. But only for Backup and Copy jobs, as offloads are not covered. That's currently a blind spot for us.

[wishr]

Got it, thanks for the additional information.

[tommy.oshea]

Good Morning Everyone,

I have been working from a corporate position on a project to get all 40 of our locations using Veeam, and pushing backups to AWS Object storage using SOBRs. However, due to various issue at multiple locations, there have been times where for whatever reason, the offload is failing, or is not syncronized, etc.
I can work on these and troubleshoot on a case by case basis, however the problem is that these occur and there's no mechanism for me to be aware of them.

Each location has their own IT Manager, who is responsible for their backups, but definitely don't have the required knowledge to notice/fix these issues. (Or they don't care)

To me, the ideal would be some kind of Veeam One monitor, or Veeam Enterprise Manager feature that I can look at everything from a centralized perspective.

Any suggestions?

[veremin]

The general issues and problems can be spotted by looking at daily SOBR report.

If you want to receive more detailed information regarding upload and offload sessions, then check the PS workaround above.

Also, it would help, if you express your ideas regarding the potential report view - this information would be useful for team responsible for Veeam ONE (and for Fedor specifically).

Thanks!

[tommy.oshea]

The problem is that I have 40 sites to monitor, and I don't want to get 40x the amount of emails in my inbox for all their regular jobs. I only want the SOBR reports. Is there a way to get only those?
If not, I guess I have to contact everyone and get them to add me to their alerts I had hoped to avoid that and just use Veeam ONE.

[wishr]

Hi Tommy,

This is something we are planning to add to VONE in the future. Thank you for the request.

[ryan.christensen]

This would also be a great item to add to Veeam Service Provider Console. We have 215 clients and will likely be moving all of them to SOBR with Azure soon depending on what Veeam v12 brings.

[Gostev]

Ryan I recommend you post your request in the Veeam Service Provider Console forum and include any service provider specific requirements. Thanks!

[yonib]

Hello,

We are a Service Provider and are looking to offload backups to object storage.
Currently we have no other way then to check the daily summary that is being sent in the VBR console.

Is there a way to monitor all the offload tasks of our customers, maybe through PS?

Thanks in advance.

[wishr]

Hi Yoni,

Welcome to Veeam R&D Forums!

I'm moving your post to an existing topic with a similar question.

Also, you are welcome to apply for Veeam Cloud & Service Providers forum. You should be a registered VCSP to get the approval.

Thanks

[mcz]

Hi guys,

we're having multiple vm's on our object storage across multiple (orphaned) buckets. Now you know that bucket A holds X TB of data but you don't really know more about it. If you have a look at the vm's in the backup properties then you see all the restore points and the size but that size is not always accurate because they don't represent the delta blocks - blocks are deduplicated on the object storage.

So in our case the capacity run out of space quite quickly and I don't know why or which vm would be responsible for that to take further actions... I would love to get a report to monitor the space consumption per vm/restore point for the specified time period.

Thanks!

[jorgedlcruz]

Hello Michael,
When you are mentioning orphaned buckets, do you mean that at some point they were part of a SOBR, but then you remove them from those SOBR? So you have a few Object Storage Repositories with backups there, and you are looking to obtain granular details per VM consumption on those buckets?

I know that Veeam ONE has the Workload Protection History Report which is going to get better and will offer transferred data to Capacity Tier, etc.

But that will be v12, and it does not give you much info on Object Storage Repositories alone. We might have some workaround using PowerShell or calling some modules, let me double-check, this might be an interesting use case to add to VONE anyways.

[mcz]

Jorge, your assumption is right. They were part of SOBR at a certain point but were replaced with a new one (for instance, when you need a new bucket for the immutability). In general, there is a need for "how many bytes does this single restore point consume", no matter if it's on the object storage, on ReFS, on XFS, etc. So I would expect something like "total bytes" and "consumed bytes". You just have to define where to add that one needed vbk

[wishr]

Hi Michael,

This request is absolutely reasonable - this is for sure, however, there are some specifics. For object storage VBR contains the data about deduplication and amounts of space consumed and saved, but this is not the case with deduplication handled by file systems of on-premises storage. You may get more information about the latter in the following topics: 1, 2 (requires access to the SP forums). We are aware of both scenarios and are planning changes for the future to provide solutions for them. However, I would not expect the changes for handling ReFS/XFS scenario to arrive in the short term.

I hope that helps! Thanks.

[mcz]

Thanks Fedor! Interesting, why is there no "easy way" to extract that information using ReFS/XFS? I'm just curious...

[wishr]

Simply, there is no interface allowing us to pull that information. Which means we have to write some internal logic on the VBR side to do the calculation on our own.

[mcz]

oh, that's crazy... I would have expected it to be part of a filesystem, but my expectations don't count here

[wishr]

Yes, when I was researching this topic for the first time about a year and a half ago, I got surprised as well! Anyway, we have a few ideas on how to get it sorted, but these will not be a part of V12, unfortunately.

[TonioRoffo]

I'm a bit stumped myself.

We use Datto RMM for monitoring, and using the event log to trigger alerts/tickets.

There's a LOT of events which work perfectly to monitor backup jobs, however, for SOBR offload, there's only event 40002 (Veeam MP), and those entries do not indicate what job the failure is linked to:

The storage management session for repository SOBR Backup has finished: Failed
Repository name: SOBR Backup
Job type: Offload
Sesson result: Failed
Mode: CopyNewBackupFiles

I can't track which job the error belongs to. So even with succesful 40002 events later on, I have to create a ticket for the one failed offload & let a colleague manually check if the backups worked later on.

We need more information for monitoring.

Thanks