I'd like to clarify something:
I will have to Completely restart the backup server as the host it is on keeps losing connection to the backup server.
your Veeam Backup server is a VM that sits on a ESXi host. The ESXi host occasionally loses the connection to the VM, is that correct?
I am also logged in as the Local Host of the VM to see if it is a communication issue between the backup server and a user but no still fails even when using the local host account.
Please elaborate on that. You are not able to login into the VM even if you use local admin account, right?
I am unsure where I would get logs from Backup and Replication to help Diagnose the issue, I feel as if this is something simple I am missing.
Please check this