Still merge issues with 9.5

Availability for the Always-On Enterprise

Still merge issues with 9.5

Veeam Logoby mkretzer » Sat Dec 03, 2016 3:03 pm

Case 01897612 and now 01987409

In 9.5 we still have the issue that backup merges fail with messages like "Failed to process method {Transform.Patch}: Application is shutting down."

The problem is that it happens every month or so. Registry settings, upgrade to 9.5 did not help. Not using per-VM seems to help a little bit...

We start to grow desperate because with every new feature we want to use we worry that it will get worse.

Markus
mkretzer
Expert
 
Posts: 330
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Still merge issues with 9.5

Veeam Logoby Gostev » Sat Dec 03, 2016 7:04 pm

I looked up the support cases. First case looks to be closed due to no responses from you, and the second one was just opened a few days ago. Please keep working with our support on this new one to get the issue resolved fully, otherwise even version 19.5 will still have it ;) The issue is not exactly wide spread, so we may never be able to catch it in our QC labs either (and it might not be even a Veeam issue to start with). Thanks!
Gostev
Veeam Software
 
Posts: 21442
Liked: 2362 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Still merge issues with 9.5

Veeam Logoby mkretzer » Sat Dec 03, 2016 8:11 pm

We had the reg setting implemented and waited for the issue to occour. It takes some time...
mkretzer
Expert
 
Posts: 330
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Still merge issues with 9.5

Veeam Logoby Gostev » Sat Dec 03, 2016 8:23 pm

That reg setting was nothing special, basically a timeout to kill data mover after in case of inactivity. Default is 1 hour, but as I understand the issue still happened when you bumped it to 8 hours. The inactivity is caused by the fact that data mover is waiting for the OS to acknowledge write to disk - most basic WriteFile() API call gets stuck writing a block of data to disk (never hears back from the OS that the data was written). It is very unlikely to be a Veeam bug - so far, all signs point to either some hardware issue (backup server or storage), or a Windows OS bug.

What Windows version are you using for backup server, any chance to upgrade one? If not, it might make sense for our support to open a support case with Microsoft on your behalf. They have debug versions of DLLs they normally use to analyze these sort of issues.
Gostev
Veeam Software
 
Posts: 21442
Liked: 2362 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Still merge issues with 9.5

Veeam Logoby mkretzer » Sun Dec 04, 2016 9:36 pm

The remote Repo server is running 2012R2. It happens on two totally different storages both attached to that server, one uses a Dell MD3600f and the other a Fujitsu DX60. No FC switching. Different HBAs.

The issue happens on both storages at the same time so it is most likely a Windows or Veeam issue. Would you recommend we upgrade to windows 2016 (we could do a reinstall, there are no other services on that system)?

The interesting thing is that the next copy always seems to fix this...
mkretzer
Expert
 
Posts: 330
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Still merge issues with 9.5

Veeam Logoby Gostev » Sun Dec 04, 2016 10:28 pm

Sure, it is worth a try - even though I am more inclined to think the issue is in hardware or drivers.
Gostev
Veeam Software
 
Posts: 21442
Liked: 2362 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Still merge issues with 9.5

Veeam Logoby mkretzer » Mon Dec 05, 2016 7:30 am

Gostev,

so you really think we should replace the server? Even if the storages are FC connected? With two different HBAs?
Whouldn't that cause NTFS or Disk errors which would show up in event log?

Markus
mkretzer
Expert
 
Posts: 330
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Still merge issues with 9.5

Veeam Logoby JoshuaPostSAMC » Mon Dec 05, 2016 3:17 pm

How much RAM does your backup server have and how large are the incrementals you are merging? I had lots of problems with SQL Transaction logs and merge/compact and support was having a hard time nailing it down. Finally I upgraded the server from 16 GB to 32 GB, although it usually looked like it had plenty free, and most of my problems have gone away.
JoshuaPostSAMC
Expert
 
Posts: 117
Liked: 16 times
Joined: Thu Jul 30, 2015 7:32 pm

Re: Still merge issues with 9.5

Veeam Logoby mkretzer » Mon Dec 05, 2016 4:34 pm

Backup server has 128 GB, remote repo has 384 GB. I do not think this is the issue :-)
mkretzer
Expert
 
Posts: 330
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Still merge issues with 9.5

Veeam Logoby dgallego » Wed Jun 28, 2017 8:42 am

Hi, ┬┐Did you find a solution?

Regards
dgallego
Lurker
 
Posts: 1
Liked: never
Joined: Wed Jun 28, 2017 8:41 am
Full Name: Daniel Gallego


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 32 guests