Veeam Database Gets Corrupted Repeatedly

[aakash.rajwani]

Case ID: 07407191

Hello,

We have installed 4 VM instances earlier and every one of them was flawlessly deployed. But in a new instance that we created on a new Hyper-V host, we are unable to make it work. The Hyper-V host has NVME SSDs, RAID5, based on Intel VROC. We are hosting 4 VMs on this host. We have configured 5 backup jobs - one for each VM and one for OS backup of the host itself. There are 5-6 File copy jobs too which copy files from various VMs from other hosts too.

Every time we reinstall Veeam (the latest version), the first backup job might work. After that, the jobs start failing. We get errors like:

Task failed unexpectedly

Failed to copy item '...' Error: An existing connection was forcibly closed by the remote host

Waiting for the next task
The SQL Server machine hosting the configuration database is currently unavailable. Possible reasons are a network connectivity issue, server reboot, heavy load or hot backup. Please try again later. Error: An existing connection was forcibly closed by the remote host
Exception while writing to stream Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.


After these errors, when we restart Veeam, it will either not connect at all or we will start getting Database corruption errors of Postgres. When we tried using SQL server, the error logs start getting generated and our entire C drive (500GB) got full in 1 day!

We are unable to use the server at all because of this.

[Dima P.]

Hello aakash.rajwani,

What B&R version are you using? Did I get this right: the database engine is installed locally within the B&R machine? Thank you!

[aakash.rajwani]

Hello Dima.P,

We are using Veeam B&R 12.2.0.334. The database is created on the same machine, yes.

[aakash.rajwani]

Hello aakash.rajwani,

What B&R version are you using? Did I get this right: the database engine is installed locally within the B&R machine? Thank you!

We are using Veeam B&R 12.2.0.334. The database is created on the same machine, yes.

[aakash.rajwani]

Hello Dima P.

Can you please look into this?
We started backing up the VMs on this host via Veeam on another host and it is working fine.

Is there an issue with Veeam on VROC?

[Dima P.]

Hello aakash.rajwani,

Sorry for keeping silence and thank you for updating the thread! I've shared your case with QA team for a review, meanwhile please keep working with our support team and let us know how the investigation goes!

[aakash.rajwani]

Hello Dima P.,
We are using the Community Edition so the Support thread was closed without any action. I hope you can check it with your QA team. If Intel VROC is the challenge, it must be updated in the KB.

[Dima P.]

The issues are likely caused by the configuration database connectivity issues with B&R itself. For testing purposes try disabling all the AV / firewalls on the machine prior installing the B&R Community Edition and then perform a few tests with the AV / firewalls shut down. Thank you!

[aakash.rajwani]

Hello Dima P,

We have tried that. We took a vanilla OS, disabled FW, Hyper-V and Veeam.

[Gostev]

Try installing Veeam on another hardware then. There are no known issues like that ever with the product itself so it must be something environment specific like misbehaving storage, RAID controller, RAM etc. resulting in silent data corruption. The fact that you have the same issues also when using SQL Server only proves the issue is with the server itself. Given the environment-specific nature of the issue we will not be able to provide further assistance with the issue in this forum. Thanks