Comprehensive data protection for all workloads
Post Reply
MadAsToast
Influencer
Posts: 15
Liked: never
Joined: Jan 13, 2011 9:56 am
Full Name: Chris Hibbert
Contact:

Sage Line 500 Backups & Replication

Post by MadAsToast » Sep 26, 2011 7:29 am

Morning All,

We are experiencing issues with backups and replication on Sage line 500 - we have tried both application aware and non application aware (block level).

Sage Line 500 is hosted on one virtual machine, MS SQL database on a second virtual machine.

The problem we are experiencing is identical on both application aware and not, and whether we do backups/replication on either VM - The end user loses connection to the database from the client, we get the following error in the log file:

Code: Select all

[08S01]:[[Microsoft][SQL Native Client]TCP Provider: An existing connection was forcibly closed by the remote host.]:iebuildquery:iestmt_opencurs:table scheme.stockm
We have spoken to our Sage support guys, however it seems their stock reply to anything in depth is "network issues" which as we run a full reporting Cisco network we know the network is running at about 7% utilisation without any issues.

If anyone has any insight into backups/replication on a sage system I would appreciate any advice, I can rely on out of hours backups and replications but I'm fairly sure this shouldn't be happening especially on block level.

TIA

MadAsToast.

Gostev
SVP, Product Management
Posts: 24787
Liked: 3520 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Sage Line 500 Backups & Replication

Post by Gostev » Sep 26, 2011 8:54 am

Hi Chris, the only suspect here would be long VMware snapshot operation, but those problems would result connection to timeout instead, and here the connection is forcibly (actively) closed by SQL Server. Something is actually actively closing the connection, and you say this even happens if you do crashconsistent backups (no app-aware processing or VSS freeze). I have never seen something like this, and too suspect some network problems actually...

MadAsToast
Influencer
Posts: 15
Liked: never
Joined: Jan 13, 2011 9:56 am
Full Name: Chris Hibbert
Contact:

Re: Sage Line 500 Backups & Replication

Post by MadAsToast » Sep 29, 2011 8:25 am

Not sure I agree with the network problems. Perhaps I need to supply a little more information.

The Veeam server is connected to the storage through two fibre channel controllers connected to fabric switches, the SAN holds all the data again connected to the fabric switches.

The SQL database server AND the Sage Line 500 servers are both VM machines and tied to sit on the same physical host, therefore the communication between these servers NEVER hits the physical network, only ever connect through a vSwitch.

The disconnection of client only ever happens when the backup is running and that as stated before is both Application aware AND block level. I cant see how this will be network related as the only network being used is a virtual network to communicate between the two virtual servers!

I agree the long snapshot could have something to do with it, I'm more inclined to agree that this might be connected to the issue, HOWEVER the disconnects happen almost IMMEDIATELY after the backup/replication has been started, so I'm not convinced that the length of time on the snapshot is causing the issue. Perhaps there is a issue on the actual disk utilisation on the snapshot being taken and when the snapshot is "in use" but I'm slightly sceptic.

I'm open to any and all suggestions on this one as currently I'm having to schedule outside office hours and outside backup time replications, once I have the "window" to do the initial replication.

TIA

Chris

ThomasMc
Expert
Posts: 293
Liked: 17 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: Sage Line 500 Backups & Replication

Post by ThomasMc » Sep 29, 2011 9:18 am

We run Sage 200 with a similar setup and although I can snapshot it fine(Backup/Replicate) its one of the only things I can't vMotion during business hours due to a similar issue we get when we do it, and when we spoke to Sage Support funny enough they said it was a Network Issue lol(Cisco kit throughout and Solarwinds monitoring it all), this info probably won't really help your issue though

Vitaliy S.
Product Manager
Posts: 22976
Liked: 1555 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Sage Line 500 Backups & Replication

Post by Vitaliy S. » Sep 29, 2011 11:32 am

MadAsToast wrote:I'm more inclined to agree that this might be connected to the issue, HOWEVER the disconnects happen almost IMMEDIATELY after the backup/replication has been started
Chris, have you tried creating snapshot manually through vSphere Client to check if it has a similar impact on your VMs?

tsightler
VP, Product Management
Posts: 5418
Liked: 2240 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Sage Line 500 Backups & Replication

Post by tsightler » Sep 29, 2011 7:01 pm

Is the problem occurring during the snapshot of the Sage server, or the SQL server? I've seen this issue on Dell server with certain Broadcom NIC's whenever there is high load (a snapshot might be created similar behavior to high load). I think it was caused by TCP Offload.

Ah..here the Microsoft KB article:

http://support.microsoft.com/kb/942861

The note even mentions that the error may occur when you "replicate SQL databases". Obviously they're not referring to Veeam replication, probably just due to the load involved. Might not be the same issue, but it's an easy solution to try.

tsightler
VP, Product Management
Posts: 5418
Liked: 2240 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Sage Line 500 Backups & Replication

Post by tsightler » Sep 29, 2011 7:12 pm

BTW, the article at http://blogs.msdn.com/b/mssqlisv/archiv ... issue.aspx has some great resources for troubleshooting intermittent connectivity issues. It specifically mentions the TCP Offload (TCP Chimney Offload) issue I mentioned, as well as a couple of other known causes of intermittent connectivity causing timeouts that really should just be retries.

Post Reply

Who is online

Users browsing this forum: EdgarRicharte and 30 guests