SSH Session and log file issue?

[theirpuppet]

Environment:
Windows Server 2022
PSVersion: 5.1.20348.1366
Veeam Console: 11.0.1.1261 P20220302
OpenSSH 8.9.1.0
User Role in Veeam: Administrator

When I logon to the Windows Console, open Powershell and run: connect-vbrserver -server myserverl .. everything works. Also, as that user, I am able to open the Veeam Console and interact with Veeam as expected.

When I logon to the Windows Server via SSH, I invoke powershell via the CLI and then I run 'connect-vbrserver'. No matter which options I try, I always get this error:

Code: Select all

Connect-VBRServer : Cannot open log for source 'Veeam Backup'. You may not have write access.
At line:1 char:1
+ Connect-VBRServer -server myserver
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], InvalidOperationException
    + FullyQualifiedErrorId : System.InvalidOperationException
	

I have tried:
Connect-VBRServer -server myserver
Connect-VBRServer -server myserver -User myuser -Password mypass
Connect-VBRServer -server myserver -Credential $mycreds
Connect-VBRServer -server myserver -User anotheruser -Password anotherpass


Unfortunately, "C:\Users\myuser\AppData\Local\Veeam\Backup\VeeamPowerShell.txt" is not getting any logs. It does exist .. but not getting anything from these attempts to invoke Connect-VBRServer in the SSH session. NOTE: Logs are generated when I invoke connect-vbrserver from the Windows Console session. So this behaviour is limited to the SSH session.

My initial thoughts are that something is missing from the SSH environment ($env) relevant to that log file? This is causing connect-vbrserver to fail.

[david.domask]

Hi @theirpuppet,

I've never tried this, but I assume the SSH scope is limited by design with Microsoft's OpenSSH implementation. Can you even edit the file with like nano (install it via chocolatey)? I would assume it won't.

Can I ask though, why do you want to do this via SSH? Is it to do the access via a non-Windows machine?

[theirpuppet]

Hello, I'm sure that SSH is limited and that Veeam didn't test for this usage case. My particular reason (for using SSH) is rather simple: we use Rundeck to automate a lot of activities. Because Veeam doesn't provide API access to SQL Backups, we've deployed a Windows Rundeck node to automate Veeam restores, etc. On that Rundeck node, we've installed the Veeam Console in order to make use of the Veeam PowerShell Modules for things like Get-VESQLDatabase. And Rundeck server accesses the Rundeck nodes via SSH, because WinRM is not our favourite flavour of remoting.

How this all works is rather simple. Rundeck Server uses SSH to connect to the Rundeck node. In that SSH session, it simply uses powershell to invoke a script like: "powershell.exe -File /path/to/script". The script is nothing special and works when run via an RDP / Console session. It just doesn't work when run via an SSH session.

As mentioned, I believe the problem is that Connect-VBRServer (or a library it utilises) is attempting to write to "C:\Users\myuser\AppData\Local\Veeam\Backup\VeeamPowerShell.txt". In order to determine the location of that file, it could use something like $filePath = "C:\Users\$($env:username)\AppData\Local\Veeam\Backup\VeeamPowerShell.txt" ...

Now, $env:username is available, so (if I'm correct about the nature of the problem) it's utilising a different BUILT-IN ENVIRONMENT variable. But the Veeam PowerShell Modules are provided as dll assemblies, thus I have no way to dig into them to see what the problem actually is.

So, I'm asking if anyone has successfully utilised Veeam PowerShell modules via SSH? If so, they likely encountered and resolved this issue - or did something else that worked around the problem entirely.

[david.domask]

Hi @theirpuppet,

Thank you for the background, but I guess it doesn't change the situation much from my post.

I'm not going to dig into the DLL right now for this as I'm still mostly curious if it's just if you can edit the file at that path within the shell opened over SSH or not.

I actually doubt it's about environmental variables, but a simple test is just to see if Microsoft's OpenSSH implementation can parse those. If it cannot, I'm still not certain we're actually using environmental variables in the code specifically so much as Microsoft's OpenSSH not handling it.

I tried installing Microsoft's OpenSSH but for some reason on my lab server it errors out on install, so later when I have time to try a fresh server I'll take a peek, but not sure when that will be.

However, please note that our ability to support this since it involves the OpenSSH implementation is going to be quite limited since it's something that requires quite a bit of research, and the public documentation for this is very limited from what I can see, so I'm not sure we can consider this a fully supported scenario.

Do you know, is the Powershell invocation with this using Powershell 5 or is it a later version? If that's the case, it's known that PS 6/7 cannot be used right now for controlling Veeam cmdlets: powershell-f26/veeam-11-powershell-modu ... 75751.html

I'm not sure if they're calling PS 6/7 with Microsoft's OpenSSH, and if they are, then I'm afraid this is not going to work for the time being.

[theirpuppet]

Thanks, I'll try to answer each question

I'm not going to dig into the DLL right now for this as I'm still mostly curious if it's just if you can edit the file at that path within the shell opened over SSH or not.

Yes, I have full perms as SSH is just providing me a simple CMD.com shell running as the user, like any other process. I have tested this from the CLI. Also that file does exist and is being used when Connect-VBRServer is invoked by powershell.exe via a Console / RDP session.

I actually doubt it's about environmental variables, but a simple test is just to see if Microsoft's OpenSSH implementation can parse those. If it cannot, I'm still not certain we're actually using environmental variables in the code specifically so much as Microsoft's OpenSSH not handling it.

I can confirm that at least $env:username is available:

Code: Select all

PS C:\Users\myuser\AppData\Local\Veeam\Backup> $env:username
myuser

We have PS Core installed, but it is invoked as pwsh. So, I'm 100% certain that I'm using PS 5.1. The $PSVersionTable was provided in the original post.

I tried installing Microsoft's OpenSSH but for some reason on my lab server it errors out on install, so later when I have time to try a fresh server I'll take a peek, but not sure when that will be.

We're running v8.9.1.0 from https://github.com/PowerShell/Win32-OpenSSH and it was a standard MSI install.

However, please note that our ability to support this since it involves the OpenSSH implementation is going to be quite limited since it's something that requires quite a bit of research, and the public documentation for this is very limited from what I can see, so I'm not sure we can consider this a fully supported scenario.

Feel free to ask me any q's about configuration. We've only modified a couple lines of the default config (for security reasons).

Do you know, is the Powershell invocation with this using Powershell 5 or is it a later version? If that's the case, it's known that PS 6/7 cannot be used right now for controlling Veeam cmdlets: powershell-f26/veeam-11-powershell-modu ... 75751.html

As specified above and in the original post: it's definitively PS 5.1. The rundeck job is invoking powershell.exe, not pwsh.exe. When I run it manually, I'm also invoking powershell.exe to get a powershell session. PS Core is not involved or I would get a different error: yada yada yada Connect-VBRServer not found yada yada

[david.domask]

Got it, thanks for the answers @theirpuppet.

To be honest, I cannot reproduce the issue you had with the same version. I connected from SSH on MacOS as the local administrator and was able to successfully import the Veeam PS module without incident.

Code: Select all

administrator@DDOM-VEEAM-RB4 C:\Users\Administrator>powershell                                                                                                    
Windows PowerShell                                                                                                                                                
Copyright (C) Microsoft Corporation. All rights reserved.                                                                                                         
                                                                                                                                                                  
PS C:\Users\Administrator> Import-Module Veeam.Backup.Powershell                                                                                                  
WARNING: The names of some imported commands from the module 'Veeam.Backup.Powershell' include unapproved verbs that might make them less discoverable. To find   
the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.                  
PS C:\Users\Administrator> Get-VBRLicenseAutoUpdateStatus                                                                                                         
False                                                                                                                                                             
PS C:\Users\Administrator> Get-VBRJob                                                                                                                             
                                                                                                                                                                  
Job Name                  Type            State      Last Result  Description                                                                                     
--------                  ----            -----      -----------  -----------                                                                                     
Backup Copy Job 1         VMware Backu... Stopped    Warning      Created by DDOM-VEEAM-RB4\Administrator at 10/5/2022 6:19 AM.                                   
corrupt-test              VMware Backu... Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/13/2022 3:36 PM.                                  
vmware-ffi-cap-bb         VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\guy at 8/29/2022 11:14 PM.                                          
filtertest                VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/1/2022 5:15 PM.                                   
hvbackup                  Hyper-V Backup  Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/13/2022 10:31 PM.                                 
template-sub-folder       VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/9/2022 4:36 PM.                                   
linux-backup              VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 1/6/2023 2:55 PM.                                    
session-test              VMware Backup   Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 9/27/2022 8:04 AM.                                   
vbr-vss-test-backup       VMware Backup   Stopped    None         Created by DDOM-VEEAM-RB4\guy at 9/1/2022 4:56 AM.                                            
vss-console-test          VMware Backup   Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 12/7/2022 1:38 PM.                                   
Backup Copy Job 2         VMware Backu... Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 10/27/2022 5:24 AM.                                  
test nas backup           File Backup     Stopped    None         Created by DDOM-VEEAM-RB4\Administrator at 9/30/2022 12:40 AM.        

I also tried with non-local admin, and I could not reproduce. Account Backup User is set as a Backup Administrator in Veeam and also in Administrators group on Windows:

Code: Select all

backupuser@DDOM-VEEAM-RB4 C:\Users\backupuser>powershell                                                                                                          
Windows PowerShell                                                                                                                                                
Copyright (C) Microsoft Corporation. All rights reserved.                                                                                                         
                                                                                                                                                                  
PS C:\Users\backupuser> Connect-VBRServer -Server Localhost                                                                                                       
PS C:\Users\backupuser> Get-VBRJob                                                                                                                                
                                                                                                                                                                  
Job Name                  Type            State      Last Result  Description                                                                                     
--------                  ----            -----      -----------  -----------                                                                                     
Backup Copy Job 1         VMware Backu... Stopped    Warning      Created by DDOM-VEEAM-RB4\Administrator at 10/5/2022 6:19 AM.                                   
corrupt-test              VMware Backu... Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/13/2022 3:36 PM.                                  
vmware-ffi-cap-bb         VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\guy at 8/29/2022 11:14 PM.                                          
filtertest                VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/1/2022 5:15 PM.                                   
hvbackup                  Hyper-V Backup  Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/13/2022 10:31 PM.                                 
template-sub-folder       VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 12/9/2022 4:36 PM.                                   
linux-backup              VMware Backup   Stopped    Success      Created by DDOM-VEEAM-RB4\Administrator at 1/6/2023 2:55 PM.                                    
session-test              VMware Backup   Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 9/27/2022 8:04 AM.                                   
vbr-vss-test-backup       VMware Backup   Stopped    None         Created by DDOM-VEEAM-RB4\guy at 9/1/2022 4:56 AM.                                            
vss-console-test          VMware Backup   Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 12/7/2022 1:38 PM.                                   
Backup Copy Job 2         VMware Backu... Stopped    Failed       Created by DDOM-VEEAM-RB4\Administrator at 10/27/2022 5:24 AM.                                  
test nas backup           File Backup     Stopped    None         Created by DDOM-VEEAM-RB4\Administrator at 9/30/2022 12:40 AM.

So I have to assume it's something on your user account(s) that you're connecting with. If you try from a remote server with the Veeam Console installed with the same user and launch the Veeam Powershell console, does it work?

[theirpuppet]

And there we have something interesting! My user is not an Administrator on the local system. As soon as I added them to the local Administrators group .. it worked.

1. add user to local Administators group
2. logon via ssh
3. execute 'powershell'
4. connect-vbrserver -server servername

works great!

can you try with your user not being part of the windows' local Administrator group?

[david.domask]

Hi @theirpuppet, glad it worked! Keep in mind that there may still be nuances introduced by this element, so if issues with your schema do come up, test with the remote console connection first and see if it reproduces outside of the OpenSSH element.

For the test, I can already tell you it will fail on my box because this is currently a requirement. I will ask that the User Guide for the cmdlets be a bit more clear, but Administrator and Backup Administrator role are needed.

Hopefully it can be made more granular in the future for simple reports, but I hope your deployment here works out well with the current requirements.

[theirpuppet]

but [Windows] Administrator and [Veeam] Backup Administrator role are needed.

I beg to differ. The Administrators group does not appear to be an absolute requirement:
The original post illustrated that Connect-VBRServer works fine when the Windows logon session is via RDP or Windows Console. It only fails when the Windows logon session is initiated via SSH.

To reiterate, the user is not part of the Windows Administrator group and yet Veeam PowerShell modules work fine via RDP/Console. They only fail under SSH. However, Connect-VBRServer does indeed work via the SSH logon session once the user is part of the Windows Administrator group.

So it seems we have a solution - albeit a very poor one (for security) - and I thank you for that. All of this will be a non-issue once the API has matured (and the explorers' functionality is added).

[david.domask]

Well, as you wrote, for full functionality, it is required There are a lot of nuances to this, but much of the cmdlets right now require administrator functionality, to the point that for practical purposes, it is a requirement.

But this is tangential, and the point on more nuanced security is not missed. (that is, do not misunderstand me to say that I think this is fine and said and done. Far from it!) But you are very welcome, and hopefully your project goes well.

[david.domask]

Just an update for anyone else who wants/needs to try this route.

After internal discussion, we cannot offer support for use of the Veeam Powershell Module over OpenSSH at this time. It will be considered for support in future releases, but no ETA on if or when that might happen.  

If you have a need and can accept no support when OpenSSH is included, try the steps in this thread, and if there are issues, try the same commands without SSH involved and see if it works.