Why Do Storage Snapshots Keep The Host Locked?

[FECV]

This may be a dumb question, or i am just not thinking it though, but why to backup jobs that are using storage snapshots keep the host locked? As an example i have a server with several large disks and multiple backup jobs to backup up each disk in a separate job. Everything runs fine, but once a job is running and is backing up from a storage snapshot, VBR does not seem to touch the server any more. After its done it deletes the storage snap shot, removes any previous backup jobs per retention policy, then finishes. However my other jobs can not run until the excising job is done, because the server "is locked by another task". Is this really necessary? Once its handed off to the storage appliance, why can it release the server so other jobs can use it?

[david.domask]

Hi @FECV,

Can you maybe share a screenshot of the full text from the UI on what you're seeing? (remember to edit out any private information)

"is locked by another task" sounds like Veeam's scheduler holding off on this, but tasks usually aren't towards a host, they're towards a specific machine. I'm a little unclear on the sequence here, but as I understand it, you're seeing:

1. Job A starts processing VM's 1, 2, 3
2. Job B starts and tries to process VM 2
3. Job B queues this task saying VM 2 is locked by another task

Am I correct or is it really the ESXi host itself that is reported as locked?

[FECV]

Thank you for your reply.

In this case i have ServerA a file server with several very large data stores attached to it. (I forgot to add this important info to my original post that i am dealing with an Veeam Agent For Windows job here. Sorry)
I have several jobs for ServerA that run every night to backup the disks separately.
For example Job 1 for ServerA will backup one disk, and Job 2 for Server A will backup another disk, and so on.
The Jobs use storage snapshots, and mostly run just fine.

What i was trying to inquire in my post/question is while Job 1 is running, Job 2 is paused waiting for Job 1 to finish. In my case it seems like the host is out of the question when the backup from the storage snapshot takes over, so why keep the host locked?

So Job 2 will display output like the following while Job 1 is running.
--3/30/2024 1:05:13 AM :: Queued for processing at 3/30/2024 1:05:13 AM
--3/30/2024 1:05:13 AM :: Machine is locked by another task

Job 1 will show something like this at
--3/23/2024 3:58:32 AM :: Starting off-host backup
--3/23/2024 3:58:33 AM :: Queued for off-host processing at 3/23/2024 3:58:33 AM
--3/23/2024 3:58:33 AM :: Required backup infrastructure resources have been assigned
--3/23/2024 3:58:38 AM :: Using off-host backup proxy Backup Proxy for Disk 9
--3/23/2024 3:58:38 AM :: Importing snapshot set to the off-host backup proxy Backup Proxy
--3/23/2024 4:01:56 AM :: Drive (A:\\\)(off-host) (75 TB) 3 GB read at 267 MB/s [CBT]
--3/23/2024 4:04:33 AM :: Finalizing
--3/23/2024 4:07:15 AM :: Incremental backup created
--3/23/2024 4:07:18 AM :: Network traffic verification detected no corrupted blocks
--3/23/2024 4:07:18 AM :: Processing finished at 3/23/2024 4:07:18 AM

[david.domask]

Thanks for the clarifications -- if possible, can you open a case for this one? I discussed with my colleagues from our Veeam Agent for Windows (VAW) team and they would have expected a different message if if was truly the multiple VAW jobs.

Please collect logs for the affected job as well as the jobs which run at same time and report the issue, and open a case. Please post the case number here when created.

By the way, is the proxy itself (backup proxy in your example) being backed up also in your environment?

[FECV]

What might the expected message be? Curious as i thought i had been seeing expected behavior.

I have opened a case for this. Case # 07209259

I only have one VBR/Proxy server. It is physical and it functions as my VMware Backup Proxy and my Agent Backup Proxy. I do have an agent job backing up the VBR server but only the OS drive, none of the other drives that are my backup repositories. This backup job is not running at the same time as the other jobs.

[david.domask]

Thanks for opening the case and sharing the number Frederick.

The message I was told we ought expect is "another job is running", but it was also told that the messaging might have been changed and that a quick check of the logs would be best, so hopefully we should have a full answer soon

[FECV]

Yeah, the initial response from support is that what i am seeing is expected behavior. Which i expected. So i will go back to my initial question is what is the need to have resource locked by another job running. Again since i am using storage snap shots, the host is now out of the picture, from what i can see. I am trying to understand if there is a technical reason or if it was just done in programing, and may be worth a feature request. Why keep other jobs waiting if not needed.

[david.domask]

Hi Frederick,

https://helpcenter.veeam.com/docs/backu ... ml?ver=120

See Step 6 here; basically, even with Backup from Storage Snapshot, the Veeam Agent is still working on the production machine and handling backup operations, so any other jobs trying to start a backup process will queue while the first job is finishing it's operations. Remember it's not just the reading of the volume(s), the agent is handling a few other tasks as well from the production machine also.

So right now it's less about the specifics of the backup and more just in general Veeam will not process the same machine by two jobs at the same time.