-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
using gMSA through Veeam Security Domain fails
Hello there,
i'm currently testing the usage of group managed service accounts for guest procesing.
so far first tests when dooing ah backup with gMSA with VMs that are in the same Domain as the VBR (proxy) servers is working as expected.
But trying to backup ah server from ah different Active Directoy Domain does not work!
First "issue" you run into how you add the Security Desciptors from the backup-domain to the other domains... but i think i got this sorted out, especially interesting is the Domain Admin group, it should work the way that i've created an gMSA in the productive domain, added there an domain-local group from the productive domain and this domain-local group you can add the computer accounts of the proxys from the backup domain!
in veeam i'll just added then the productive\gmsa account... but using this that way does not work... i even tried to add only "gmsa" as account, but there's a syntax check inside veeam that only allows domain\gmsa or gmsa@domain... ok
the job right now has following error:
Failed to prepare guest for hot backup. Details: Failed to connect to guest agent. Errors:
'Cannot connect to the admin share. Host: [server.prod.local]. Account: [prod\gmsa_veeam].
Failed to perform safe logon
Failed to create a process token for account prod//gmsa_veeam$
Win32 error:The user name or password is incorrect.
Code: 1326
Cannot connect to the admin share. Host: [192.168.1.1]. Account: [prod\gmsa_veeam].
Failed to perform safe logon
Failed to create a process token for account prod//gmsa_veeam$
Win32 error:The user name or password is incorrect.
Code: 1326
the double slashes looks ah little bit strange to me... its also almost the same in the log of the proxy
the error code is slightly different with 1355 and there is an additional line inside the logs:
Win32 error:The specified domain either does not exist or could not be contacted.
(anonymized usernames/domains and ips here)
Case-Nr. 05889607
i'm currently testing the usage of group managed service accounts for guest procesing.
so far first tests when dooing ah backup with gMSA with VMs that are in the same Domain as the VBR (proxy) servers is working as expected.
But trying to backup ah server from ah different Active Directoy Domain does not work!
First "issue" you run into how you add the Security Desciptors from the backup-domain to the other domains... but i think i got this sorted out, especially interesting is the Domain Admin group, it should work the way that i've created an gMSA in the productive domain, added there an domain-local group from the productive domain and this domain-local group you can add the computer accounts of the proxys from the backup domain!
in veeam i'll just added then the productive\gmsa account... but using this that way does not work... i even tried to add only "gmsa" as account, but there's a syntax check inside veeam that only allows domain\gmsa or gmsa@domain... ok
the job right now has following error:
Failed to prepare guest for hot backup. Details: Failed to connect to guest agent. Errors:
'Cannot connect to the admin share. Host: [server.prod.local]. Account: [prod\gmsa_veeam].
Failed to perform safe logon
Failed to create a process token for account prod//gmsa_veeam$
Win32 error:The user name or password is incorrect.
Code: 1326
Cannot connect to the admin share. Host: [192.168.1.1]. Account: [prod\gmsa_veeam].
Failed to perform safe logon
Failed to create a process token for account prod//gmsa_veeam$
Win32 error:The user name or password is incorrect.
Code: 1326
the double slashes looks ah little bit strange to me... its also almost the same in the log of the proxy
the error code is slightly different with 1355 and there is an additional line inside the logs:
Win32 error:The specified domain either does not exist or could not be contacted.
(anonymized usernames/domains and ips here)
Case-Nr. 05889607
-
- Product Manager
- Posts: 14824
- Liked: 3075 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hello,
the syntax check exists because I wasted some hours of troubleshooting during beta testing. The format is required and not "just for the sake of it"
Best regards,
Hannes
the syntax check exists because I wasted some hours of troubleshooting during beta testing. The format is required and not "just for the sake of it"
the Guest interaction proxy must be in the production domain. Or do you have a trust relationship?accounts of the proxys from the backup domain!
I'm currently working with tech writers to replace the "should" with a "must"user guide wrote:both the backup proxy and the target machine should have access to the domain controller to obtain the gMSA password.
Best regards,
Hannes
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hi Hannes,
Proxys are in trust relationship, as the BP recommends.
But even there it's somewhat tricky to use gmsa, but as described theoretically it should be possible that way .
I did saw some security domain implementations with Veeam where there is also used users from these restricted domain, honestly not something I would do personally , as my paranoid security me didn't want to even have (User) Accounts from the security domain popping up in network traffic on production domains... But that's ah different discussion anyway...
Proxys are in trust relationship, as the BP recommends.
But even there it's somewhat tricky to use gmsa, but as described theoretically it should be possible that way .
I did saw some security domain implementations with Veeam where there is also used users from these restricted domain, honestly not something I would do personally , as my paranoid security me didn't want to even have (User) Accounts from the security domain popping up in network traffic on production domains... But that's ah different discussion anyway...
-
- Product Manager
- Posts: 14824
- Liked: 3075 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hello,
hmm, with domain trust, it should work (that was not mentioned initially as far as I see).
During gMSA testing I figured out a couple of test steps. Do they all work for you?
1) use the "Test-ADServiceAccount" cmdlet on all involved machines (VM, guest interaction proxy, VBR server)
- The command can be installed with Add-WindowsFeature RSAT-AD-PowerShell
- this command must return "true". otherwise AAIP will fail
2) Check whether computer can select gMSA account in services.msc
- Try to select a user and restart a service
- To switch back, you need the console https://www.wojcieh.net/change-windows- ... l-account/
and restart service and change back to Local System
- On a machine without permissions, you get an error like this one
3) check which machines can retrieve passwords
- https://serverfault.com/questions/58297 ... e-accounts
Best regards,
Hannes
hmm, with domain trust, it should work (that was not mentioned initially as far as I see).
During gMSA testing I figured out a couple of test steps. Do they all work for you?
1) use the "Test-ADServiceAccount" cmdlet on all involved machines (VM, guest interaction proxy, VBR server)
- The command can be installed with Add-WindowsFeature RSAT-AD-PowerShell
- this command must return "true". otherwise AAIP will fail
Code: Select all
Test-ADServiceAccount svc_gmsa04
- Try to select a user and restart a service
- To switch back, you need the console https://www.wojcieh.net/change-windows- ... l-account/
Code: Select all
sc config VeeamDeploySvc obj=lab\administrator password=PASSWORD!
- On a machine without permissions, you get an error like this one
Windows could not start the Veeam Installer Service service on Local Computer. Error 1069: The service did not start due to a logon failure.
3) check which machines can retrieve passwords
- https://serverfault.com/questions/58297 ... e-accounts
Code: Select all
Get-ADServiceAccount -Identity svc_gmsa04 -Properties PrincipalsAllowedToRetrieveManagedPassword
Best regards,
Hannes
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Sorry thought it was clear with trust, else you wouldn't even be able to add proxy from the Management-Backup domain to the domains where the to be backup'd VMs are in!
I gonna test your described steps....
I also thought maybe through the domain trust it's necessary to "install" the gmsa, normally, as far as I understood this, it shouldnt be needed with gmsa .. this was only ah thing with the older normal "MSA"...
I gonna test your described steps....
I also thought maybe through the domain trust it's necessary to "install" the gmsa, normally, as far as I understood this, it shouldnt be needed with gmsa .. this was only ah thing with the older normal "MSA"...
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
i guess the question now is what directions of trust is needed... right not its an one-directional trust only... prod domain trusts the mgmt domain (as described in bp: https://bp.veeam.com/vbr/Security/Secur ... e-of-trust)
as described ah couple of times now, but i'll write down now the complete list of groups involved (using the domainnames from the bestpractices example):
Assumption is, we would like to make Backups of the Domain Controllers from production.local!
production.local:
gMSA-Account -> MemberOf Domain Admins (espacally interestting as this is ah GLOBAL group and no objects from trusted domains can be inserted directly!!)
in gMSA_Account though we can add a domain-local-group from the production.local and in this group there you can add all the computeraccounts from veeam vbr/proxys inside (again with an trusted group, shouldn't matter i guess "mngt\VeeamProxyServers") = PrincipalsAllowedToRetrieveManagedPassword
now in theory these computeraccounts should be allowed to retrieve the password of the production\gMSA-Account!
i'll found ah following post that descripes this too:
https://stackoverflow.com/questions/583 ... ed-domains
I have used gMSA accounts across a domain trust. The gMSA principal needs to be a group in the same domain, but as long as the group is type Domain Local, you can add computers from the other domain as members to that group, and they are then able to retrieve the password successfully.
i did some tests now...
HannesK wrote: ↑Feb 22, 2023 7:03 am
During gMSA testing I figured out a couple of test steps. Do they all work for you?
1) use the "Test-ADServiceAccount" cmdlet on all involved machines (VM, guest interaction proxy, VBR server)
- The command can be installed with Add-WindowsFeature RSAT-AD-PowerShell
- this command must return "true". otherwise AAIP will failCode: Select all
Test-ADServiceAccount svc_gmsa04
first test didn't succeed, when trying to "test-adserviceAccount production\gmsa-account" from VBR Server in mngt.local domain i'll get an error:
Cannot find an object with identity: 'production\gMSA-Account' under: 'DC=mngt,DC=local'.
in mngt.local side this is simply not possible to use the gMSA Accounts from production.local domain, as there is no trust and hence you can't select the production domain...
this one now gets interessting again!HannesK wrote: ↑Feb 22, 2023 7:03 am 3) check which machines can retrieve passwordsCode: Select all
Get-ADServiceAccount -Identity svc_gmsa04 -Properties PrincipalsAllowedToRetrieveManagedPassword
with just "Get-ADServiceAccount -Identity svc_gmsa04 -Properties PrincipalsAllowedToRetrieveManagedPassword" its the same output as with the test-serviceaccount, the command tries to find the account in the mngt.local domain... -> cant find it...
but there is an additional parameter -server, and with that set to the production.local domain the gMSA Account is retrieved correctly!
Code: Select all
Get-ADServiceAccount -Identity gmsa-account -Server production.local -Properties PrincipalsAllowedToRetrieveManagedPassword
DistinguishedName : CN=gmsa-Account,CN=Managed Service Accounts,DC=production,DC=local
Enabled : True
Name : gmsa-Account
ObjectClass : msDS-GroupManagedServiceAccount
ObjectGUID : d3446df3413-0961-43dd9-806c-4a55ffsa2bc73d
PrincipalsAllowedToRetrieveManagedPassword : {CN=gTrustedServers,OU=TEST,DC=production,DC=local}
SamAccountName : gmsa-Account$
SID : S-1-5-21-16634533820-776332457623-456333347-5348
UserPrincipalName :
there is no "-Server" parameter with the test-adserviceaccount available...
-
- Product Manager
- Posts: 14824
- Liked: 3075 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hello,
QA confirmed what I wrote in my initial answer: the guest interaction proxy has to be in the production domain.
I will ask tech writers to clarify the text in the user guide.
Best regards,
Hannes
QA confirmed what I wrote in my initial answer: the guest interaction proxy has to be in the production domain.
I will ask tech writers to clarify the text in the user guide.
Best regards,
Hannes
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Ok thx for clarification... I also come after some other tests to this conclusion... We've also tried some different configurations where the gMSA from the management domain should have been used...
I've also found something related to this ... When using gMSA for SQL transaction Log backups ... The SQL server account also need s to be allowed to retrieve password. Didn't read this in the docs, might also just missed it ..
One other thing regarding gMSA, i was ah little sad when I found out that for adding Proxys etc no gMSA can be used... Only normal account can be selected...
I've also found something related to this ... When using gMSA for SQL transaction Log backups ... The SQL server account also need s to be allowed to retrieve password. Didn't read this in the docs, might also just missed it ..
One other thing regarding gMSA, i was ah little sad when I found out that for adding Proxys etc no gMSA can be used... Only normal account can be selected...
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Apr 10, 2023 5:00 pm
- Full Name: Renee Riley
- Contact:
Re: using gMSA through Veeam Security Domain fails
Could you expand on this please? I'm not sure what you mean when you say "adding Proxys etc no gMSA can be used". Are you referring to choosing specific proxy servers on the Guest Processing tab vs. using "Automatic Selection"? Thank you.
BTW, I'm currently working through this on case #05985777.
Thank you, Renee.
-
- Product Manager
- Posts: 14824
- Liked: 3075 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hello,
my understanding of Stephan's statement is "when you add the proxy machine as a managed Windows server, then username / password is required".
Best regards,
Hannes
my understanding of Stephan's statement is "when you add the proxy machine as a managed Windows server, then username / password is required".
Best regards,
Hannes
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hi, yes as Hannes stated, i meant that in theory it should also be possible to use the the gMSA's as "connection" users für the Proxy Component management/installation!
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Sep 21, 2023 9:51 am
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hi Are there any news regarding this problem, we do have exactly the same issue
-
- Product Manager
- Posts: 14824
- Liked: 3075 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hello,
and welcome to the forums.
Which one? We discussed a few things in this topic. Adding a Windows server requires username / password. That's the current design.
Best regards,
Hannes
and welcome to the forums.
Which one? We discussed a few things in this topic. Adding a Windows server requires username / password. That's the current design.
Best regards,
Hannes
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Sep 21, 2023 9:51 am
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hi Hannes,
sorry for the late response. We finally managed to figure out our problem. a group policiy blocked the access, which has been deleted now. Everything is working fine now. We had the same error as described in the first message.
sorry for the late response. We finally managed to figure out our problem. a group policiy blocked the access, which has been deleted now. Everything is working fine now. We had the same error as described in the first message.
-
- Novice
- Posts: 3
- Liked: 1 time
- Joined: Nov 02, 2021 4:15 pm
- Contact:
Re: using gMSA through Veeam Security Domain fails
I've been trying, and failing, to get transaction logs to backup using gMSA for weeks now, and this is the first time I've encountered this little bit of rather crucial info in my searches.
I'm not sure how or why this would be required, but it's not a minor detail.
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
the sql transaction log agent "runs" on the sql server and connects to the "veeam repository" hence the sql needs to be allowed to use the gMSA...
-
- Novice
- Posts: 3
- Liked: 1 time
- Joined: Nov 02, 2021 4:15 pm
- Contact:
Re: using gMSA through Veeam Security Domain fails
Oddly enough, the Log Pruning service doesn't seem to be storing anything on the Log Shipping server (assuming I've set it up correctly), it's just using it as a proxy. Everything is still being stored on the primary backup server/repository.
I suppose this is the way it's supposed to work? Although it makes me wonder what the advantage of Log Shipping servers is, as the logs were previously stored on the main repository without the need for any proxies.
Unfortunately, at this point, I'm no longer sure which issues are the result of recently upgrading from v11 to v12 (and some settings that have been added and aren't properly set), and which are due to the change from a single domain admin user account to gMSA.
I suppose this is the way it's supposed to work? Although it makes me wonder what the advantage of Log Shipping servers is, as the logs were previously stored on the main repository without the need for any proxies.
Unfortunately, at this point, I'm no longer sure which issues are the result of recently upgrading from v11 to v12 (and some settings that have been added and aren't properly set), and which are due to the change from a single domain admin user account to gMSA.
-
- Novice
- Posts: 4
- Liked: never
- Joined: Aug 17, 2020 3:38 pm
- Full Name: Stephan
- Contact:
Re: using gMSA through Veeam Security Domain fails
Stumbled upon your thread having the same problem.
Found out that veeam doesn't like it when the accounts are part of the Protected User group. Any attempt to put the gMSA backing up our DCs into this group ends up with exactly the same error. Seems veeam doesn't like kerberos at all and keeps using NTLM!? Which if true would be pretty bad... Maybe someone can confirm this?
Regards
Stephan
Found out that veeam doesn't like it when the accounts are part of the Protected User group. Any attempt to put the gMSA backing up our DCs into this group ends up with exactly the same error. Seems veeam doesn't like kerberos at all and keeps using NTLM!? Which if true would be pretty bad... Maybe someone can confirm this?
Regards
Stephan
-
- Service Provider
- Posts: 293
- Liked: 45 times
- Joined: Jun 30, 2015 9:13 am
- Full Name: Stephan Lang
- Location: Austria
- Contact:
Re: using gMSA through Veeam Security Domain fails
Hi, gmsa are not working in protected users group because of Microsoft..... Gmsa's rely on Kerberos constraint Delegation, what would be blocked if in the protected users group... The gmsa also not necessarily needs to be in there... To administer these accounts only tier 0 admins should have access... And in the hosts where it's allowed to retrieve the password only limited admins should have access...
It might also be ah could idea to split them up into multiple accounts for different tasks.
Best regards and nice holidays
Steve
It might also be ah could idea to split them up into multiple accounts for different tasks.
Best regards and nice holidays
Steve
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Oct 26, 2023 11:20 pm
- Full Name: Tomas Echepetelecu
- Contact:
Re: using gMSA through Veeam Security Domain fails
In our case we had to ensure we added the gmsa to the allow to log on as a service GPO as we apply CIS Level 1 policies to our environment.
Who is online
Users browsing this forum: lohelle and 83 guests