I want to start off with thanks to Veeam support as this was resolved with case #02086181.
I am posting this because I am hoping someone might be able to help me understand what is going on with the VMware architecture that caused this issue to occur.
The setup for this one is that I have on one of my VMware vSphere ESXi 6 U2 hosts, a Windows 2012 R2 Virtual Machine that hosts a SQL Server 2012 instance. I had just run the upgrade from Windows Server 2012 to the 2012 R2 edition a few weeks before. The ISO file for the install was located on a datastore on our NAS. At some point during the day, the NAS went offline, causing the datastore to be unavailable. Since I had completely forgot to dismount the ISO file, the cdrom device of the VM was now attached to a ghost ISO file. This caused snapshot functionality to not work. The Veeam Replication would start a snapshot, then I think it timed out, leaving an orphaned snapshot, then continued to do this something like 42 times.
Once Veeam support helped me identify the issue, I was able to apply most of the steps in this forum post https://communities.vmware.com/thread/481804?tstart=0
to resolve the issue.
I know this is a VMware question, but still hoping someone here might be able to explain why would the datastore becoming unavailable cause snapshots to fail
? It seems like something that is either a glitch in VMware error handling, or something that administrators need to be very cautious about, because it could have catastrophic consequences related to disaster recovery capability.
I will post most of this question in the VMware forums also, and will update this with any relevant information.
Thanks in advance for any explanations of this issue.