Backup Copy Job - Merge failure

[bcharp82]

Case# 01100202

I need some help since it seems like my VEEAM support rep doesn't seem to have any help ideas.

My backup copy job for our file server VM consistently fails once it reaches the point in time where it merges the full backup file. The job will get stuck at the file merge. The job contains a single VM. This VM contains an antivirus server, firewall server, and copy location for our physical file server (1.5 tb). I have recreated the job multiple times and it will run through the incrementals with no problems but fails once a merge is to be performed.

The storage target is a Buffalo NAS with RAID 10, dual core processor, and 2 gb of RAM. I thought this might be the issue but our other backup copy jobs containing multiple VM's are processing just fine. Some of the other jobs that are successful are about the same size or even larger. The incremental size of copy job in question is about 20gb on average while other backup copy jobs have incrementals that are larger at 35gb in size. Based on these observations, I believe that the storage is not the issue.

To make sure the Buffalo NAS doesn't get overworked, I have scheduled the backup copy jobs to run in a fashion where no more than two jobs are processing at the same time.

I'm not sure where to go from here. The backup copy feature used to work for this VM....

[foggy]

Brandon, does the job get stuck or actually fails (what is the error message then?)? Have you tried the same job to some other storage?

[bcharp82]

Foggy,

The job isn't actually stuck. But it takes 4-6 hours to a for the progress meter to increase by 1%.

Here are the errors in the email report:

Code: Select all

"Failed to merge full backup file. Terminated at 11/17/2015 3:16:05 PM
Failed to create synthetic full restore point 11/8/2015 6:00:00 AM
Failed to generate points Error: The specified network name is no longer available. Failed to read data from the file [\\TS5800DDDE\Backups$\Backup Copy Job - File Server bkup via App5 F\Backup Copy Job - File Server bkup via App5 F2015-11-07T060000.vbk]."

The error suggests that the NAS becomes unavailable but our network monitor never goes off. No other jobs get this error and we have other jobs that are larger.

I have not tried another storage because we currently don't have any alternate storage spaces. We've budgeted for more storage in 2016 but we have to wait right now.

[foggy]

Please continue investigating with support. Job debug logs should contain information about what operations take so long as well as the actual cause of errors.

[PTide]

Hi,

looks like a network issue to me - have you tried running "ping" while merge is happening to see if any timeouts occur? Please take a look at this post. Also how long is your backup copy chain?

Thank you.

[bcharp82]

looks like a network issue to me - have you tried running "ping" while merge is happening to see if any timeouts occur? Please take a look at this post. Also how long is your backup copy chain?

If this was a network issue, this should affect other jobs as well. However, the problem is limited to just this one backup copy job. I've scheduled the backup copy windows around differently to see if the job behaves differently but this has not changed the outcome. I have other larger jobs that take 12 hours to complete but they never get the same error.

Our network monitor pings the NAS device every minute and creates a notification is the ping response time is greater than 1000ms. If I ping the NAS from the backup server right now, the majority of the response times is 1ms with the highest observed latency of 24ms. That's with two backup copy jobs running at the moment.

This job is configured to keep 7 restore points.

[bcharp82]

I hope this works differently in the new version...

[foggy]

New version will allow you to avoid merges at all by utilizing the backup copy active full.

I'd still suggest to escalate the case for deeper investigation of the issue.