My original case was #05688029.
I reproduced the logged error message using the following steps:
- Install a standalone windows server 2019 into a hyper-v vm.
- Enable file sharing (if needed).
- Opent Event Viewer, Application log.
- Create VBR backup job, Windows computer, Managed by server. Add vm's IP as individual host. Add credentials for the vm's Administrator account. Backup mode doesn't matter (can be file-level targeting a nonexistent path, or an empty second volume).
- Start job, wait for rescan and agent install to finish, and then watch the backup progress.
- Observe that the error event is logged very early, before "Required backup infrastructure resources have been assigned".
Through trial and error I've narrowed it down to the "Enable application-aware processing" part. Even with "transactional logs: copy only" it still happens. If I disable application-aware processing, the error is not logged. But this did not help to locate the cause.
Using Process Monitor, I was unable to capture the place where the 'access denied' happens. There were no such events. This made me think that it is DCOM-related.
This time, instead of typing the full error message into web search, I only typed 'IVssWriterCallback', intending to check microsoft documentation on what component it belongs to. Surprisingly, the top results were all old articles describing exactly what I posted. They said that it happens with third party backup software, and that it is a dcom permission issue. Specifically, the
Local Access permission for
NETWORK SERVICE. The articles were sloppy and just instructed to add it to the default setting for all dcom processes, which I found excessive. However, it does work, which helps narrow things down. Note that a system reboot was always required to apply dcom permission changes, at least during my testing.
I dug deeper, and found that
CallFailureLoggingLevel's default setting causes all dcom call errors to be hidden. Setting it to 1 reveals
The machine-default permission settings do not grant Local access permission to the COM Server application C:\Program Files\Veeam\Endpoint Backup\Veeam.Endpoint.Service.exe with APPID Unavailable to the user NT AUTHORITY\NETWORK SERVICE SID (S-1-5-20) from address LocalHost (Using LRPC) running in the application container Unavailable SID (Unavailable).
Putting the two together, it seems that Veeam.Endpoint.Service.exe registers a COM callback in VSS, presumably for all the vss writers to report back. When the "System Writer" (provided by CryptSvc) tries to call back, it gets denied and the callback doesn't happen.
According to
CoInitializeSecurity, a client or server application is able to declare its own custom security rules. If it doesn't, the default system-wide ones are used.
I examined the programming of all Veeam executables that reference CoInitializeSecurity, and found that every single one of them uses the lazy defaults with a NULL security descriptor. The Microsoft article
Calling CoInitializeSecurity in a Secure Manner demonstrates how to declare access to builtin groups, if the Veeam devs want to do this properly, and not ask of the user to expose the entire DCOM system to Network Service. I do not know what the exact list of SIDs to declare is, but at most it's Self, System, Administrators and Network Service.
(I still do not know why running the job in Managed by Agent mode doesn't run into this error. Presumably, the agent launches its worker process with the credentials configured in the job, which in my case is a member of the Administrators group and hence already covered by the default setting.)