Snapshots take a while to be created

[EBoucq]

Hi,

Correct me if I'm wrong but I thought that a backup job with the guest quiescence enabled triggers a VMware snapshot "without the VM memory".
On a VMware 6.0 infra with a Veeam B&R 9.5 U3, I have a few VMs whose snapshots (during a Veeam Backup) take more than 2 minutes to be created, exactly like if the memory was included. In that same job, other VMs don't have this problem. When I do a test directly from the vCenter (snapshot without memory), I don't have this issue and the snapshot takes a few seconds to be created.
That's why I'm wondering if the issues wouldn't be on Veeam side...
Anyone already had this problem ?
Thanks.
Eric

[PTide]

Correct me if I'm wrong but I thought that a backup job with the guest quiescence enabled triggers a VMware snapshot "without the VM memory".

You are correct on this one - Veeam should not snapshot VM memory state if quiescence has been selected.

When I do a test directly from the vCenter (snapshot without memory), I don't have this issue and the snapshot takes a few seconds to be created.

During the test, did you select "Quiesce guest filesystem" checkbox in vSphere UI?

Thanks!

[EBoucq]

Hi,

No, during the manual snapshot test, I didn't select "quiesce guest file system". Should I ?

Thanks,
Eric

[PTide]

Yes, please do so. The main idea behind my proposal is to test whether quiescing via VMware will also take that long. I suspect that the in-guest VSS writers might be the culprit.

Thanks!

[Regnor]

Is your vSphere 6.0 on a current patchlevel?
I remember having similar problems which were solved patching the hosts.
Also check if on events inside the VMs regarding VSS timeout and if you have other backup jobs or similar tasks running.

[EBoucq]

Ok, so when I do a snapshot test from the vCenter, with "Quiesce guest file system" enabled, it's super slow.
The version of vSphere is 6.0.0 4192238

[PTide]

That is, it's worth checking VSS writers events as per Regnor's advice.

Thanks!

[EBoucq]

Thanks PTide.

I checked the VSS writers and didn't see any errors. FYI, it's a Sybase database in the VM so no VSS writer.
But, what I see in the system log are "system failed to flush data to the transaction log" warnings...

[PTide]

Such events are usually followed by a more detailed description, such as "device busy" or "A device which does not exist was specified", and ID (might be 140, 57,or some other). To me it seems that the error does not really have anything to do with Veeam, and that its probably worth opening a ticket with MS while awaiting here for other ideas from fellow users.

Thanks!

[evilaedmin]

Thanks PTide.

I checked the VSS writers and didn't see any errors. FYI, it's a Sybase database in the VM so no VSS writer.
But, what I see in the system log are "system failed to flush data to the transaction log" warnings...

Are general VMware snapshot operations on reasonably busy VMs performing to expectations? Any general storage latencies which may be exacerbated by snapshot create/delete operations? Snapshot operations can amplify the amount of write IO. The Windows events you mention could be a symptom of underperforming storage.

Latencies are funny in that at 35 ms everything may appear to be normal, but just a few more % IO can cause 35ms to turn into 100, and everything falls apart. (made up example).

[EBoucq]

Thanks for all your replies.
I'll keep searching as some VMs don't have this issue although having a snapshot created at the same time....

Eric