Comprehensive data protection for all workloads
Post Reply
mintyevertonian
Enthusiast
Posts: 27
Liked: never
Joined: Nov 12, 2013 9:55 am
Full Name: fintan quinn
Contact:

Backup Copy Job Issues

Post by mintyevertonian »

Error: Timed out waiting for the backup repository resource to become available for backup integrity check


I am trying to setup a new backup copy job to a repository

when the job starts i am getting this error

The repository has 4 jobs running to it concurrently and I have changed the max number to 5

I have restarted the repository and the veeam server . the other backup copy jobs seem to be running fine to this repository

any thoughts?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy Job Issues

Post by foggy »

Fintan, please include the support case ID for this issue, as requested when you click New Topic. Thanks!
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: Backup Copy Job Issues

Post by lando_uk »

I'm also getting this on all my copy jobs this weekend.

They are scheduled for a integrity check on the last Sunday for every month.

30/03/2014 21:12:11 :: Error: Timed out waiting for the backup repository resource to become available for backup integrity check

I disabled the jobs, changed the check to a different day and restarted the jobs, but they are still all running and producing this error every few mins.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy Job Issues

Post by foggy »

What is the max number of concurrent tasks for this repository and how many tasks do you run concurrently when getting this error? Have you already contacted technical support with this?
tomas.odehnal
Influencer
Posts: 16
Liked: 2 times
Joined: Oct 02, 2012 11:34 am
Full Name: Tomas Odehnal
Contact:

Re: Backup Copy Job Issues

Post by tomas.odehnal »

Hello,

I've got the same problem. Since Sunday 4 of 5 copy jobs return this error. I tried to run just one job at time and it didn't help.
Support case ID 00542915.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup Copy Job Issues

Post by Vitaliy S. »

How many tasks do you have configured on this repository? Did anything change since Sunday?
tomas.odehnal
Influencer
Posts: 16
Liked: 2 times
Joined: Oct 02, 2012 11:34 am
Full Name: Tomas Odehnal
Contact:

Re: Backup Copy Job Issues

Post by tomas.odehnal »

There are 5 jobs, one of them finished ok. Nothing changed since Sunday.
We have single server deployment. I tried to restart the server, but that didn't help.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup Copy Job Issues

Post by Vitaliy S. »

Ok, but how many concurrent tasks are allowed on this repository right now? Can you please double-check?
tomas.odehnal
Influencer
Posts: 16
Liked: 2 times
Joined: Oct 02, 2012 11:34 am
Full Name: Tomas Odehnal
Contact:

Re: Backup Copy Job Issues

Post by tomas.odehnal »

There are four concurrent tasks allowed on both source and destination repositories.
muditha
Lurker
Posts: 2
Liked: never
Joined: Feb 18, 2014 10:52 am
Full Name: Muditha Gayan
Contact:

Re: Backup Copy Job Issues

Post by muditha »

Hi guys, I'm also having this problem, in my case first time the job ran successfully, from the next time it began to give this error, but this was happened only to one job, all the other copy jobs were successfully finished so far
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy Job Issues

Post by foggy » 1 person likes this post

Is there a chance that the time the job starts to fail at falls into the denied data transfer period (the one configured at the Schedule step)? QC guys report that there's currently a known issue when the job starts failing with the said error if backup health check is scheduled to be performed during the denied period. Apparently, support can provide a private fix for this, so please contact them directly.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup Copy Job Issues

Post by Vitaliy S. »

@Tomas, hmm... given that you have 4 tasks configured at target repository and even one job running does not resolve this, I would recommend to review job logs with your support engineer for further investigation.

@Muditha, do you point other jobs to the same repository? How many jobs do you run at the same time? Have you tried running only this very job?
muditha
Lurker
Posts: 2
Liked: never
Joined: Feb 18, 2014 10:52 am
Full Name: Muditha Gayan
Contact:

Re: Backup Copy Job Issues

Post by muditha »

@Vitaliy : Thank you, Yes I tried it that way as well, run only that job, increased concurrent tasks, but still getting this error. And I have another question, Can we schedule backup jobs using windows 2012 scheduler ? If we can could you please share a link or document how to configure this ? thanks you
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy Job Issues

Post by foggy »

Please see my post above regarding this error. As to your scheduling question, you can search this forum for the examples of a simple PowerShell script used to trigger the job and schedule it via Windows Task Scheduler.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Copy Job Issues

Post by veremin »

Otherwise, you can take the commandline command shown in the summary page of job settings and schedule it via Windows Scheduler to run at whatever basis you want to. Thanks.
tomas.odehnal
Influencer
Posts: 16
Liked: 2 times
Joined: Oct 02, 2012 11:34 am
Full Name: Tomas Odehnal
Contact:

Re: Backup Copy Job Issues

Post by tomas.odehnal »

@foggy, you are right. If I change the schedule to "Any time", the job starts. I'll notify the support engineer.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy Job Issues

Post by foggy »

Thanks for confirming. This should be addressed in the next product patch.
tomas.odehnal
Influencer
Posts: 16
Liked: 2 times
Joined: Oct 02, 2012 11:34 am
Full Name: Tomas Odehnal
Contact:

Re: Backup Copy Job Issues

Post by tomas.odehnal » 1 person likes this post

I can confirm support provided temporary fix which solved the problem for me. Thanks again.
guillermo.lozano
Enthusiast
Posts: 69
Liked: 10 times
Joined: Feb 04, 2013 4:35 pm
Full Name: Guillermo Lozano
Contact:

Re: Backup Copy Job Issues

Post by guillermo.lozano »

Hello, This same thing happened to me right after patch #3.

I was able to resolve this by setting the time window to "Any Time"

Regards,
Guillermo.-
spgsit5upport
Expert
Posts: 221
Liked: 16 times
Joined: May 28, 2010 10:25 am
Full Name: Seb
Contact:

Re: Backup Copy Job Issues

Post by spgsit5upport »

But I do NOT want Anytime!

I want to exclude times

Seb
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Copy Job Issues

Post by veremin »

Have you had a chance to upgrade to the latest product version? The said bug should be fixed there.
Health Check process continuously fails with the "Timed out waiting for the backup repository resource to become available for backup integrity check" error during the periods when the Backup Copy job is restricted from using network bandwidth.
Thanks.
spgsit5upport
Expert
Posts: 221
Liked: 16 times
Joined: May 28, 2010 10:25 am
Full Name: Seb
Contact:

Re: Backup Copy Job Issues

Post by spgsit5upport »

Thanks, I have now updated to patch 4 (updated to patch 3 not long ago...)

Seems to be OK (does not fail, but waits for the allowed time)

Again, bug fixed without need to Support ID

Seb
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job Issues

Post by Gostev » 2 people like this post

Known issues is a different thing. Admit that you could have "fixed" this not only without opening a support case, but even without posting on the forums (by just reviewing the sticky topic in this very forum).

In the bigger picture though, bug was fixed in the recent patch only because there were plenty of support cases filed for this issue. Otherwise, it would not have been fixed until v8. It is support who tells R&D what common bugs they like to be fixed in the patch, and they judge solely based on the number of support cases for the specific issue.

That is just so you understand one of the reasons why we are strictly enforcing support case ID when people are posting about any technical issues, other than known and documented of course.
spgsit5upport
Expert
Posts: 221
Liked: 16 times
Joined: May 28, 2010 10:25 am
Full Name: Seb
Contact:

Re: Backup Copy Job Issues

Post by spgsit5upport »

So after running for 3 hours (2 h 20 min of the allowed time slot), copying one VM (large one), a backup job started, which made made the backup show:

17/06/2014 18:31:24 :: Operation was canceled by user

Well, user did not cancel it. Most likely the fact that other job wanted to process same VM did the actual "cancel"

No, I will not be opening support case, simply do not care enough about it, will rather use rsync
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job Issues

Post by Gostev » 2 people like this post

If rsync was any good, we would not have developed Backup Copy. Be very careful with rsync, as it has a bad record in our support...

Some reliability issues that come to my mind:
1. Does not handle interruptions in application-aware manner (updates to file are not transactional, so it may leave your backup file at target corrupted).
2. Does not do network traffic verification (network traffic corruption is the common issues causing backup corruption).
3. Does not to periodic health checks of backup files data to detect storage level corruption.

But most importantly, rsync is going to be really slow on large backup files, because it has no idea what blocks have changed from one pass to the next, so it has to re-read the entire file, on both ends to calculate the hashes. Also, because it reads and hashes the entire file into memory, it has a tendency to fall over on very large files. Files that are 100GB or larger can slow to a crawl. Might not be a problem for you if your backup files are quite small though.

Hope this helps!
Post Reply

Who is online

Users browsing this forum: No registered users and 163 guests