Feature request: Kerberos only authentication

[shayani]

I have discussed with Veeam support and apparently there is no way to use only Kerberos authentication with Veeam. Therefore, if you're planning to have a NTLM free environment, Veeam would be out of the equation. The following services do not seem to work in Kerberos:

• 1) File indexing
  2) File restoration

I'm starting this thread to request for the feature to be in the next release of Veeam B&R. It is high time for Veeam to catch up with the latest Enterprise security standards.

[Vitaliy S.]

Hello Shayani,

This functionality is in our high priority feature list. Based on your scenario you would need to have this type of authentication available for guest processing services (application aware processing), right?

Thank you for the FR.

[shayani]

Yes. Thanks for your reply. And please post any updates.

[dsegel]

Any updates on if/when NTLM will be going away as a requirement in Veeam? My security admin is pushing to disable it everywhere.

We're running Hyper-V if it matters.

[Gostev]

Support for Kerberos-only authentication for guest connections was added in Update 4, see What's New for more details.

[dsegel]

Unless I'm reading it wrong, it says that's for vSphere only. As I said, we're running Hyper-V.

[Gostev]

Ah, sorry. Moved to the correct forum and asked for clarification why it says vSphere only, since guest processing is not hypervisor-specific logic AFAIK.

[Gostev]

Checked with the devs and apparently unlike VMware, Hyper-V does not return FQDN of the guest - while Kerberos authentication is impossible without FQDN.

[dsegel]

Yup, I just discovered this myself when all the backups on our test system failed last night.

Veeam can't even connect to the host server, nevermind the guest VMs.

Any ideas for workarounds or other possibilities for ditching NTLM for Hyper-V in the future?

Thanks.

[hke]

There's a similar issue with the console. Administrators in the Windows Protected Users group (which makes users Kerberos-only) can't connect.

veeam-backup-replication-f2/ad-protecte ... 56276.html

[benthomas]

Checked with the devs and apparently unlike VMware, Hyper-V does not return FQDN of the guest - while Kerberos authentication is impossible without FQDN.

I'm sorry @Gostev but Hyper-V guests do return their FQDNs....
The KVP exchange exposes the guest FQDN property to the host. I've just tested and confirmed this on both WS2016 and WS2019 hosts.

[Gostev]

I looked up that email thread with the devs again, and they did mention KVP there... but, in a bad light. They said it's very unreliable, as in many labs the content of FQDN field was empty. So unfortunately, this not something we can rely on for the production code.

[HenrikS.]

Hello,

Has there been any devlopment on this FR for Hyper-V guests yet, for instance is something coming in v10?
To have guest interaction service accounts leverage Kerberos only/Windows protected user group is something that we really would like to see.

-BR

[Gostev]

Hello! There was no development, as this would require Microsoft to fix this KVP functionality in Hyper-V first. Thanks!

[HenrikS.]

Hello Gostev,

But what about PowerShell Direct?
You leverage this as a fallback to inject the Veeam Service, why is it not possible to extract the hostname from the Guest?
Quick test performed on a 2016 host:

PS C:\Invoke-Command -vmName guest -ScriptBlock {([System.Net.Dns]::GetHostByName(($env:computerName))).Hostname}
cmdlet Invoke-Command at command pipeline position 1
Supply values for the following parameters:
Credential
guest01.contoso.com

-BR

[Gostev]

PowerShell Direct is a fairly recent technology that is not available in all Windows guest OS and Hyper-V versions we support. May be in future, once we only support those versions... if Microsoft still does not fix KVP by then.

[HenrikS.]

Hello Gostev,

I'm sorry, but I find this lack of functionality to be quite severe.
Is it really so that you claim KVP broken? Exactly what part of it and how can we reproduce to file a request to Microsoft?
PowerShell Direct has been around for nearly 4 years.

I want to be solution oriented here, have you considered other options to be able to provide this functionality?

1: Implement an order of precedense for guest logon; Kerberos only, Negotiate or NTLM/Legacy where the user can first move to Negotiate before Kerberos only.
2: Get the FQDN where you can. If PowerShell Direct/KVP can be used, use it. If not, then add a field under Processing Settings that makes us able to manually add the FQDN.

-BR

[Gostev]

I'm not sure I follow your logic here, Henrik. What is the point of having Kerberos-only authentication feature, if you must still keep NTLM enabled on the network for failover purposes of processing those VMs which cannot be processed with Kerberos authentication? I mean, it's not like NOT backing up some machines is acceptable, right?

And as long as you keep NTLM available on the network, from security perspective it does not matter if you now use Kerberos for certain machines... which means implementing this functionality adds zero real value.

As a real solution, you could consider moving to vSphere, where we fully support Kerberos-only guest processing.

[HenrikS.]

Hello Gostev,

I see that the logic is a bit diffuse.
What I tried to point out is that if Veeam is unable to automatically get the FQDNs, then the administrator could feed this manually.

To find out for what VMs Veeam is unable to get the FQDNs, a solution could be to enable NTLM failover and to sort these out before you would disable NTLM all together.
I guess you could also find these by the guest processing test credentials run.

The question still remains; Could Veeam implement a way/some way/any way of letting Hyper-V users leverage Kerberos-only authentication?
Changing the backup solution is more likely an option, than to change our hypervisor platform.

-BR

[christian.naenny]

Hello experts,

In documentation for Veeam B&R 9.5U4 for VMware vSphere it is stated:
"To back up or replicate VMware vSphere VMs where Kerberos is used, you must make sure that NTLM traffic is allowed in Veeam backup infrastructure machines."
Kerberos Authentication for Guest OS Processing

In our network, NTLMv2 ist entirely disabled for all new machines with Windows Server 2016 or higher. I need to obtain a special permission from our IT Security Officer to enable NTLMv2 between the Backup Server (also Tape Server and Repository Server) and the Backup Proxies. But our IT Security department is asking by when Veeam will ditch the requirement for NTLM completely.

Is there any roadmap for such a request?

Best regards,
Christian

[benthomas]

@Gostev , any chance to get the devs to review their opinions on Hyper-V KVP?

I have not seen any stability issues in my uses of it personally, and have queried others to find their experiences are the same.
Surely this could be added as a precedence option, like trying various connection methods like Powershell Direct, or like others have said, let us manually provide the FQDNs in the backup job settings if it's so critical.

[Gostev]

Microsoft did not have any new Hyper-V releases since my post in October last year. We will certainly review this once they ship their next LTSC.

[rura_at_tivoli]

Hello
Any changes since the last post?
We just encountered this problem when adding the account used for Hyper-V host and guest-processing to the AD Protected Users group.

I see no mention of this limitation in the VBR v11 Release Notes.

[Gostev]

No changes.

[aresgodofwar30]

I have a 2019 Hyper-V host with a 2019 guest. In the guest I look at the registry key HKLM:\Software\Microsoft\Virtual Machine\Auto\FullyQualifiedDomainName and it has the correct FQDN for that VM. Why can't that be used?

[Gostev]

How do we retrieve it from the guest in a Kerberos-only network?

[StephanG]

After the Petitpotem Mitigation the application-aware processing of our CA is not working anymore.
I reply to this thread as it seems "uptodate"

The backupserver is non domain joined but the interaction proxy is. Can this work or should i just get my CA in an extra job without application-aware processing checked?
Or is it best practice to just "application-aware" backup the machines that really need it?

[Gostev]

You can disable application aware processing granularly for this VM in the existing job settings, no need to create a new one.

The best practice is to always have application-aware processing enabled for all VMs.

[AlexGre]

We ran into the same issue as result of the PetitPotam mitigation steps. After reaching out to the support and getting the answer that NTLM is required for AAP, I was quite surprised by this. Especially as NTLM has been known to be vulnerably for years, with even Microsoft themselves note that e.g. in [1]: “NTLM and NTLMv2 authentication is vulnerable to a variety of malicious attacks, including SMB replay, man-in-the-middle attacks, and brute force attacks.”

Some way to use Kerberos for AAP would really be highly appreciated, as we do plan to disable NTLM all together in the near future for our domain. As state before, maybe you could allow admins to specify a FQDN for the VM manually? Or maybe you could try to use reverse DNS to get a FQDN before falling back to NTLM (and fail AAP if it has been disabled) if none exists?

There seems to be a way to use Kerberos with IPs. Microsoft supports this since Windows 10 version 1507 and Windows Server 2016 [2]. Adding the SPN and creating the mentioned registry key on the backup server immediately allowed me to access the admin share using the IP via windows explorer, which was not possible before. However, the backup would still fail. Adding the registry key on the AD/CS server and rebooting both it and the backup server did not helper either. I’ve found another forum post [3] where someone was able to get it working tough.

However, while this is would be an ok workaround for a single VM, having to add and maintain all these additional SPNs would be quite cumbersome for all our VMs anyway. An additional way, e.g., via reverse DNS as noted before, would be highly appreciated.

[1] https://docs.microsoft.com/en-us/window ... lm-traffic
[2] https://docs.microsoft.com/en-us/window ... os-over-ip
[3] microsoft-hyper-v-f25/unable-to-perform ... 69134.html

[JPMS]

This functionality is in our high priority feature list.

Posted April 2018 but it doesn't seem to be a very high priority for Hyper-V users. Reading through this post, the answers, quite frankly, seem a little complacent.

I looked up that email thread with the devs again, and they did mention KVP there... but, in a bad light. They said it's very unreliable, as in many labs the content of FQDN field was empty. So unfortunately, this not something we can rely on for the production code.

So what are you doing about it? Veeam clearly has much better contacts with Microsoft than most of us do, has Veeam actually raised this with them? Or are you expecting them to magically fix it if they are unaware of the issues you are having? If your devs have found it unreliable, surely they have lots of details of where it hasn't worked for them that they can pass on.

NTLM has been a problem waiting to happen for some time and we've spent the weekend testing moving to a Kerberos only environment, only to find it has been a complete waste of time because Veeam still haven't addressed this requirement.