9,5 SQL DB Connection Lost AOS Client Microsoft AX

[firewall68]

Hello togehter,

we have the following problem. when we start backing up our sql servers (Microsoft AX 2012 R2 SQL 2014)
our AOS Servers loose there connections to the SQL Database.
first we thought it is only on the productiv SQL Cluster but meanwhile it is also on the single SQL Servers for Test and DEV.

In the Veeam Console everything is "Green" no issues but the AOS Servers stop the service because they have no connection and the AX Application cache is empty (affects the users in the morning when they start using AX)
Has anyone the same problem or an idea?

Maybe it has something to do when merging the snapshot disk (CBT is disabled because of NFS Shares on Netapp)

Setup:
Physical Backup Machine (HP - Gen with Veeam 9.5 (Windows 2008 R2)
enough space for backup direkt attached HP disk storage (FC)
3 Virtual Veeam Proxy Servers (Windows 2012 R2, 8 GB RAM 4 CPUs)
1 SQL for DEV and dedicated AOS Servers (virtual, Windows 2012 R2)
1 SQL for TEST and dedicated AOS Servers (virtual, Windows 2012 R2)
1 SQL CLuster for Productiv and 3 dedicated AOS Servers (virtual, Windows 2012 R2)
ESXI 5.5 actual version

Thx
Mike

[veremin]

Can you tell at what stage do you typically experience this issue? During job beginning or closer to its end? Thanks.

[firewall68]

Can you tell at what stage do you typically experience this issue? During job beginning or closer to its end? Thanks.

Hello Eremin, it is closer to the end, when the backup starts there is no error message in the event viewer on the AOS Server but after lets say the last quarter of the backup time the SQL native Client will lost the connection.
I^ve tried it today starting veeam backup with a single SQL Machine and watching the event Log on the AOS Server. Backup starts at 12:59, first Error in the eventviewer was 4 min later at 12:03, backup End 13:08 h.

The Error Stays on because nobody restarted the service on the AOS Server.

Veeam Backup Settings:
Reverse Incremental, Enable inline Data dedup, Exclude Swap file blocks, Exclude deleted file blocks, compression Optimal, Storage opt: local target, no CBT, no storage integration.
The Backup Settings Enable Application-aware processing (because of SQL Logs)

Code: Select all

Event Viewer Error on AX AOS
Event 117, Dynamics Server 02
Object Server 02:  The database reported (session 2 (-AOS-)): [Microsoft][SQL Server Native Client 11.0]Communication link failure. The SQL statement was: ""

Object Server 02:  Microsoft.Dynamics.Ax.Xpp.ErrorException: Exception of type 'Microsoft.Dynamics.Ax.Xpp.ErrorException' was thrown.
   at Microsoft.Dynamics.Ax.MSIL.Interop.throwException(Int32 ExceptionValue)
   at Microsoft.Dynamics.Ax.MSIL.Interop.ttsbegin()
   at Dynamics.Ax.Application.BatchRun.serverGetOneTask(Batch batch, String serverId) in BatchRun.serverGetOneTask.xpp:line 18
   at Dynamics.Ax.Application.BatchRun.serverGetTask(Batch batch, String serverId) in BatchRun.serverGetTask.xpp:line 10
   at BatchRun::serverGetTask(Object[] )
   at Microsoft.Dynamics.Ax.Xpp.ReflectionCallHelper.MakeStaticCall(Type type, String MethodName, Object[] parameters)
   at BatchIL.startTask()
   at BatchIL.CheckBatchJobs()
   at BatchIL.threadFunc(Object threadArg) 

Object Server 02:  The database reported (session 1 (-AOS-)): [Microsoft][SQL Server Native Client 11.0]Communication link failure. The SQL statement was: "SELECT T1.SESSIONID,T1.SERVERID,T1.VERSION,T1.LOGINDATETIME,T1.LOGINDATETIMETZID,T1.STATUS,T1.USERID,T1.SID,T1.USERLANGUAGE,T1.HELPLANGUAGE,T1.CLIENTTYPE,T1.SESSIONTYPE,T1.CLIENTCOMPUTER,T1.DATAPARTITION,T1.RECVERSION,T1.RECID FROM SYSCLIENTSESSIONS T1 WHERE ((STATUS=?) AND (SERVERID=?))"

Regards Michael

[veremin]

Then, it appears that during snapshot commit the corresponding VM looses connection temporarily and clients cannot reach it therefore.

It's not a unique problem (especially for highly transactional VMs). More information regarding it (as well as potential solutions) can be found here.

Thanks.

[firewall68]

Thx Eremin, maybe it is a VMWare problem, could you please post the URL again?
Thx Mike

[veremin]

You're right, this problem is not directly to Veeam, but more to the way snapshot technology works. Though, in vSphere 6.0 release number of improvements have been introduced for this matter, so, it might be worth updating to it, if you haven't done that already.

By the way, I've updated the link.

Thanks.