Backup Copy Jobs failing

VMware specific discussions

Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Fri Oct 25, 2013 8:28 am

Hi All,

Raised a support request also: 00466465

Most of our backup copy jobs are failing with the following error message:

Code: Select all
Error: End of file --tr:Cannot read data from the socket. Requested data size: [24]. --tr:Receive file failed --tr:Wan connection failed to run command. --tr:Wan connection failed to get target file '{32b511b1-f35c-4fd4-a29d-6548975ef031}' --tr:Sync global cache failed. --tr:Disk processor failed to init global dedup. --tr:Disk processor failed to process disk. --tr:Sync task failed to process disk.Arguments

Our backup copy job for our Exchange DAG server fails with the following error:

Code: Select all
Processing '<DAG server>' Error: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond [IP of remote Veeam server]:2500 Processing '<DAG Server>' Error: The remote procedure call failed RPC function call failed. Function name: [InvokerTestConnection]. Target machine: [IP of remote Veeam server]. Processing '<DAG server>' Error: Data transfer is currently restricted Job has failed unexpectedly


I've tried rebooting all of our Veeam servers (1 x full VBR and 1 x proxy at each end, all of which are acting as WAN accelerators) just in case, but no change.

Thanks in advance.

Dave
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

Re: Backup Copy Jobs failing

Veeam Logoby v.Eremin » Fri Oct 25, 2013 8:35 am

Hi, Dave.

Have these issues occurred only recently or you have been having them ever since you installed version 7? Did you seed target cache or let the backup copy jobs to populate it, instead?

Anyway, based on the first error description, it looks like a problem with target WAN Accelerator/cache. Have you tried to re-deploy target WAN Accelerator and/or global cache?

Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Fri Oct 25, 2013 8:55 am

Hi Vladimir,

What I did was firstly have the local and 'remote' VBR servers/WAN accelerators actually all on the local LAN, and run the BCJ's that way, to let the target WAN accelerator cache be populated in that manner. I then moved the 'remote' VBR server and WAN accelerator to their actual home, our remote site! The jobs seem to run with intermittent success, but have been failing with the described errors for over a week now (I was on annual leave, so wasn't aware until yesterday).

I've not tried to redeploy target WAN accelerator or global cache (not sure how to do the global cache anyway), because our offsite link is very slow (10mbps), and I imagine it would take an absolute age to repopulate it (we have around 40-60GB worth of CBT changes per day on our local backup jobs)?

Thanks

Dave
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

Re: Backup Copy Jobs failing

Veeam Logoby foggy » Fri Oct 25, 2013 9:07 am

Dave-Departed wrote:I've not tried to redeploy target WAN accelerator or global cache (not sure how to do the global cache anyway)

Dave, you can re-seed the WAN global cache using this procedure.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backup Copy Jobs failing

Veeam Logoby v.Eremin » Fri Oct 25, 2013 9:08 am

Yep, there is no way to re-populate global cache, apart from deleting cache or re-seeding it again. As to target WAN Accelerator, you can try to delete it, create a new one, and map it to the already existed global cache.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Fri Oct 25, 2013 11:20 am

foggy wrote:
Dave-Departed wrote:I've not tried to redeploy target WAN accelerator or global cache (not sure how to do the global cache anyway)

Dave, you can re-seed the WAN global cache using this procedure.


Hi Foggy,

Where do I copy the GlobalCache\trg folder from? Out of the 4 WAN accelerators, I only have 2 which contain the \trg folder (which I assume stands for 'target', as the local WAN accelerators contain the \src folder instead?) which are the remote/target WAN accelerators. The 2 \trg folders don't have the same content.

Thanks

Dave
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

Re: Backup Copy Jobs failing

Veeam Logoby foggy » Fri Oct 25, 2013 11:30 am 1 person likes this post

I was talking about creating the cache locally and then seeding it to remote site to avoid copying data over a slow link. You can set the backup copy job locally (as you did previously) and then just copy the cache folder from the target WAN accelerator of the local job to the remote WAN accelerator server.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Fri Oct 25, 2013 12:58 pm

Support is reporting that it appears to be down to a loss of network connectivity, which I am going to investigate, but seems unlikely, given that our connection to our remote location has been up, stable and connected for 16 days now.
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

Re: Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Thu Oct 31, 2013 9:05 am

Quick update. I have configured local Backup Copy Jobs to recreate the Global Cache, as described above, and I am just about ready to take the cache down to our remote site to reseed it. However, one of the new BCJ's still won't run/complete correctly. Any guesses which it is? That's right, it's the one which targets an Exchange 2010 DAG member!

I appear to have solved the RPC error reported in my first post on this thread, it looks like this was being caused by the network adapters being in the wrong priority within the guest OS (Server 2008 R2, FYI), ie the NIC connecting to our SAN network was top in the list, whereas the NIC connecting to our Production network was 2nd in the list. Since changing this, I haven't seen the RPC error, so I am hopeful.

However, now the BCJ relating to the Exchange VM described above, fails with "Error: Data transfer is currently restricted". It's the only one that's now failing. Anyone know what this means?

Thanks

Dave
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

Re: Backup Copy Jobs failing

Veeam Logoby foggy » Thu Oct 31, 2013 9:16 am 1 person likes this post

Just to check the first guess, are there any periods when the job must not transport data defined at the Schedule step of the wizard?
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backup Copy Jobs failing

Veeam Logoby Dave-Departed » Thu Oct 31, 2013 10:58 am

foggy wrote:Just to check the first guess, are there any periods when the job must not transport data defined at the Schedule step of the wizard?

Yes, upon further investigation, it would seem that I had specified a schedule (I must've done it on auto-pilot), and that, because it was the first run of the job, it was taking a while, and therefore hit the end of the scheduled period before completing the data transfer. I've removed the schedule, and am trying it again.
Dave-Departed
Expert
 
Posts: 167
Liked: 6 times
Joined: Wed Feb 29, 2012 3:11 pm

[MERGED] Active Interval Not Found

Veeam Logoby tomsmi » Mon Jan 30, 2017 2:56 pm

Any one stumbled upon this kind of error?
Image
tomsmi
Influencer
 
Posts: 18
Liked: 1 time
Joined: Tue Jun 07, 2011 8:26 am
Full Name: Tomasz S.

Re: Backup Copy Jobs failing

Veeam Logoby PTide » Mon Jan 30, 2017 3:49 pm

@tomsmi

Hi,

Kindly check foggy's guess and if that won't help please open a support case and post you case ID.

Thanks
PTide
Veeam Software
 
Posts: 3019
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: Backup Copy Jobs failing

Veeam Logoby tomsmi » Tue Jan 31, 2017 3:28 pm

All 24h are available for transfering data. I'll take a deeper look and eventualy open a case.
tomsmi
Influencer
 
Posts: 18
Liked: 1 time
Joined: Tue Jun 07, 2011 8:26 am
Full Name: Tomasz S.


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 15 guests