I'll add that Hyper-V VSS errors are a constant headache for me. I've got so many VMs that constantly give me warnings because it couldn't properly run the VSS snapshot process, and many even on SQL server systems, so logs never get truncated unless I do so manually. I know full well it's not a Veeam issue because I've run though it with Veeam support before, but I thought I'd add my $0.02 to the conversation.
For us, about 95% of the time, the error is:
- Code: Select all
Unable to create snapshot (Microsoft CSV Shadow Copy Provider) (mode: Veeam application-aware processing with failover). Details: Writer 'Microsoft Hyper-V VSS Writer' is failed at 'VSS_WS_FAILED_AT_PREPARE_SNAPSHOT'.
The writer experienced a non-transient error. If the backup process is retried,
the error is likely to reoccur.
--tr:Failed to verify writers state.
--tr:Failed to perform pre-backup tasks.
With this immediately following:
- Code: Select all
Make sure VM does not have 'iSCSI Software Target Storage Provider' feature installed.
Which is wholly fun because we don't use Microsoft's iSCSI Software Target Provider ANYWHERE in our systems. At all. We use the initiator on the Host systems, but not in the VMs. None of our virtual machines use anything iSCSI at all.
When I went though it with Veeam support, we never really got anywhere with it other than "it's probably a bug with Hyper-V's VSS management". The end-result suggestion being "recreate the VM metadata", basically create a new VM and attach the existing VHDs to the new VM. Sometimes that worked, sometimes it didn't. And the times that it did work, it worked without issue for maybe a week or two before coming back to throwing warnings again.
Now that it's been ongoing for probably close to 2 years, I've found some consistency in which machines will almost inevitably have this problem. They usually don't start off with the issue, but at some point, weeks or months down the road, it rears its ugly head.
These are the consistent factors:
- Version 1 VMs
- Running Windows Server 2008 R2
- Running antivirus (we use ESET File Security exclusively, but I've tested with Webroot SecureAnywhere Business Endpoint Protection and had the same result)
Although that last one with antivirus is semi-consistent. If the problem happens on a system that has had antivirus installed on it at ANYTIME in the past, it'll keep doing it. A fresh VM likely won't ever do it so long as you NEVER load antivirus on it. Uninstalling typically doesn't fix it. And even with that, running a Windows system without antivirus is just asking for trouble.
I'd honestly love to be able to use VMware rather than Hyper-V in our environment, but it's just not affordable for us yet, and manageability-wise, we're a 100% Microsoft shop, so getting people to learn and become familiar with anything different is a nightmare.
I'm glad to hear that Microsoft is revamping the Hyper-V Backup API in Server 2016. I really, REALLY hope that makes it more robust and fixes a lot of these types of issues.