SureBackup - PowerShell "access denied"

[theadamlion]

Good afternoon all, I am trying to get a PowerShell script working inside a SureBackup job - I have tested running the script manually and it works as intended, however when I go to run it via the SB job I keep getting "access denied" messages when authenticating to the VM itself. I have verifed that I can ping the masquerade IP of the VM from the VBR server and that I can even authenticate to the VM via the masquerade IP using a UNC path and the local account created on the VM in question - so far everything checks out in terms of accessibility but when I go to run the job I get this in the logs (sanitized IPs and job/server info):

[29.04.2024 13:58:50.790] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] Process is starting with the following start info:
[29.04.2024 13:58:50.790] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > WorkingDirectory = C:\Program Files\Veeam\Backup and Replication\Backup
[29.04.2024 13:58:50.790] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > FileName = C:\WINDOWS\system32\windowspowershell\v1.0\powershell.exe
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] [masqueradeIP] Connecting to remote server <masqueradeIP> failed with the following error message : Access is denied.
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] For more information, see the about_Remote_Troubleshooting Help topic.
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] At C:\ps\powershellscriptname.ps1:4 char:5
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] + $ReturnCode = Invoke-Command -Credential $CredObject -Computernam ...
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] + CategoryInfo : OpenError: (masqueradeIP:String) [], PSRemotingTransportException
[29.04.2024 14:00:05.905] <43> Info (3) [SureBackup] [JOBNAME] [ScriptTests] [Console] + FullyQualifiedErrorId : AccessDenied,PSSessionStateBroken
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] Process has been finished:
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > ProcessId = 37744
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > StartTime = 4/29/2024 1:58:50 PM
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > ExitTime = 4/29/2024 2:00:05 PM
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > TotalProcessorTime = 00:00:00.7031250
[29.04.2024 14:00:05.937] <50> Info (3) [SureBackup] [JOBNAME] [ScriptTests] > ExitCode = 1

I have also made sure the Windows Firewall is being disabled as part of the SB job and I added the masquerade IP address explicitly as a TrustedHost.

The PowerShell script being run is as follows:

Code: Select all

Param($TestVmIP)
    $ReturnCode = 1
    $CredObject = Import-Clixml -Path C:\ps\CredObject.xml
    $ReturnCode = Invoke-Command -Credential $CredObject -Computername $TestVmIP -ErrorAction Stop -ScriptBlock{
  

# create an array of paths to search through
$folders = 'E:\Foldername\*'
# create an array of file names to look for
$files = 'budget2024.xlsx', 'exec-contacts.docx', 'pw.txt', 'payroll.docx'

Get-ChildItem -Path $folders -Include $files -File |
       Get-FileHash -Algorithm MD5 | 
       Export-Csv -Path 'C:\ps\newhashes.csv' -UseCulture -NoTypeInformation

$oldHashes = Import-Csv "C:\ps\hashes.csv"
$newHashes = Import-Csv "C:\ps\newhashes.csv"
$compare = Compare-Object $oldHashes $newHashes -Property Hash, Path 
	if ($compare.SideIndicator -eq "<=" -or $compare.SideIndicator -eq "=>"){
		Exit 1
	}
	else{
		Exit 0
	}
}
exit $ReturnCode

Case #07240325 has been logged with support - if anyone has any thoughts or suggestions on this one, I would greatly appreciate them. Thanks all.

-Adam

[david.domask]

Hi Adam,

Thank you for your detailed post and for sharing the case number; as I see it, the Engineer suspects it has to do with the fact that the script will run under the SYSTEM account, while I'm guessing your tests are done with a normal user account.

https://learn.microsoft.com/en-us/sysin ... ads/psexec

As a quick test, can you try starting powershell using PSEXEC with the -s flag to run as SYSTEM (confirm with `whoami` on the new shell started by PSEXEC), then try running your script; does it work this way or you get similar errors?

[theadamlion]

David, thanks for the reply and I am about to try the manual script run now as setting the creds via the Linked Jobs section of the SureBackup job settings did not help (same "access denied" error in the logs).

I have reached out to support for clarification on how to fire off the script via the elevated PowerShell session and once I have that, i will give it a shot and report back.

Thanks David.

[theadamlion]

Just in case anyone runs into a similar issue here, the way to call the script manually from VBR and run it against the SureBackup VM's masquerade IP is:

$CredObject = Import-Clixml -Path C:\<path to xml file>\CredObject.xml #assumes you have created a secure XML credentials file

Invoke-Command -ComputerName <masquerade IP> -FilePath c:\<path on local VBR server to ps1 file> -credential $CredObject

Finally, the missing piece that solved the "access is denied" errors came from a Stack Overflow post here - https://stackoverflow.com/questions/445 ... fiederrori

"I used the command Set-PSSessionConfiguration -ShowSecurityDescriptorUI -Name Microsoft.PowerShell to add the client user on the remote machine. I was then able to communicate from the client machine to remote machine"

Sure enough after running Set-PSSessionConfiguration -ShowSecurityDescriptorUI -Name Microsoft.PowerShell on the SB VM (running in troubleshooting mode) and allowing the user defined in $CredObject, the script ran as expected.

[david.domask]

Hi Adam,

Thank you for sharing your solution -- so indeed it was on the Windows/Powershell side and looks like the answer doesn't even require too much to be added to the script