-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Backup jobs and Replication jobs sometimes run concurrently
My Backup job is scheduled to start at 7pm and is usually done by 10:30pm. I have my Replication job scheduled to start at 11:30pm to give it a little buffer, but sometimes the Backup job runs slow for some reason and the two jobs overlap, resulting in a machine possibly not replicating because it is in an active backup.
Reading some other posts got me thinking about how to overcome this and I decided on not scheduling the nightly Replication job but to take the command line and put that in as a Post Job Activity on the Backup job. So in theory my Backup job will run based on the schedule I set, and when it is finished it will kick off the Replication job.
Does anyone have any experience doing things this way? Just wondering if there are any hang-ups to expect. It is going to do the first run in this configuration tonight.
Thanks!
h
Reading some other posts got me thinking about how to overcome this and I decided on not scheduling the nightly Replication job but to take the command line and put that in as a Post Job Activity on the Backup job. So in theory my Backup job will run based on the schedule I set, and when it is finished it will kick off the Replication job.
Does anyone have any experience doing things this way? Just wondering if there are any hang-ups to expect. It is going to do the first run in this configuration tonight.
Thanks!
h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- Chief Product Officer
- Posts: 31796
- Liked: 7297 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
Darhl, daisy chaining jobs is good approach - I have not heard anyone reporting issues with that. Let us know if it worked as expected for you.
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
Well, the daisy chain worked fine, but I'm still getting errors in my backups. At least they are not related to the machine already being locked by another job. Another thing I noticed is that when daisy chaining, the 2nd job doesn't retry if it fails. Which leads me to what if the originating job fails and goes into a retry, will it try to kick off the 2nd job every time it retries?
My 2 main backup jobs are to backup the iSCSI storage and to backup the internal BladeChassis storage. The backup job then kicks off the replication job for the same datastore.
My BladeChassisStorage replication gave me these two errors: (note this one tried to run for 29 minutes)and (this one tried to run for 54 minutes)
And the iSCSI replication gave me these errors: (this one tried to run for 15 minutes)
and this error on 3 machines (each of these tried to run for 20 seconds)
Also of possible note (if it matters) the vCenter Server is one of the machines that responded with the 4th listed error.
Another noteworthy item is that the replica jobs go across a VPN to a remote site. Right now I have that configured locally (we have two internet connections) and I have a VPN built between the two for the replica jobs. This is for proof of concept before I actually move the replica jobs to our co-lo site.
My 2 main backup jobs are to backup the iSCSI storage and to backup the internal BladeChassis storage. The backup job then kicks off the replication job for the same datastore.
My BladeChassisStorage replication gave me these two errors: (note this one tried to run for 29 minutes)
Code: Select all
Backing file "nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm1_name>/<vm1_name>-flat.vmdk" Failed to backup file "nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm1_name>/<vm1_name>-flat.vmdk" to "/vmfs/volumes/4a799858-07483c89-2b15-00145e2b33c0/VeeamBackup/<vm1_name>(vm-110)/replica.vbk" Replicate failed Client error: ChannelError: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Code: Select all
Backing file "nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm2_name>/<vm2_name>-flat.vmdk" Failed to backup file "nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm2_name>/<vm2_name>-flat.vmdk" to "/vmfs/volumes/4a799858-07483c89-2b15-00145e2b33c0/VeeamBackup/<vm2_name>(vm-733)/replica.vbk" Replicate failed Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [33872]. Cannot backup file in the service console mode. File: [nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm2_name>/<vm2_name>-flat.vmdk]. VBK: [veeamfs:*:/<vm2_name>-flat.vmdk@/vmfs/volumes/4a799858-07483c89-2b15-00145e2b33c0/VeeamBackup/<vm2_name>(vm-733)/replica.vbk]. RBK: [veeamfs:9:/<vm2_name>-flat.vmdk@/vmfs/volumes/4a799858-07483c89-2b15-00145e2b33c0/VeeamBackup/<vm2_name>(vm-733)/2009-09-08T234610.vrb]. Server error: An existing connection was forcibly closed by the remote host Backup request cannot be processed. Source file path: [nfc://conn:<vcenter_server_name>,nfchost:host-32,stg:datastore-736@<vm2_name>/<vm2_name>-flat.vmdk].
Code: Select all
Completing current object backup process RemoveSnapshotfailed, snapshotRefsnapshot-1748, timeout 3600000. Failed to login to "<vcenter_server_name>" by SOAP, port 443, user "domain\administrator", proxy srv: port:0 Unable to connect to the remote server No connection could be made because the target machine actively refused it <vcenter_server_IP>:443
Code: Select all
Validating object Failed to login to "<vcenter_server_name>" by SOAP, port 443, user "domain\administrator", proxy srv: port:0 Unable to connect to the remote server No connection could be made because the target machine actively refused it <vcenter_server_IP>:443
Another noteworthy item is that the replica jobs go across a VPN to a remote site. Right now I have that configured locally (we have two internet connections) and I have a VPN built between the two for the replica jobs. This is for proof of concept before I actually move the replica jobs to our co-lo site.
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- Chief Product Officer
- Posts: 31796
- Liked: 7297 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
These issues would be best to take to our support, they may have seen them before in other environments and may be able to advice with resolution. By the way, do you have custom web service port configured on your vCenter server by any chance?
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
Nope, no custom web service port. I'll email the above post to support.
Any thoughts about the daisy-chaining and why the 2nd job didn't retry?
Thanks,
h
Any thoughts about the daisy-chaining and why the 2nd job didn't retry?
Thanks,
h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- Chief Product Officer
- Posts: 31796
- Liked: 7297 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
We should be able to see the reason for not retrying from the full log files (from Help | Support Information).
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
Which log should I look at? There is only 1 from last night at 8:45pm, the next closest is from 2 weeks ago (8/29). Are my logs not configured to generate properly?
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- VP, Product Management
- Posts: 6035
- Liked: 2860 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
You need to look in the directory for the user the service is running under if this is different than what you are logged in as. This confused me at first as well.
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backup jobs and Replication jobs sometimes run concurrently
Got it. That is confusing!
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
Who is online
Users browsing this forum: Bing [Bot], gojain, josema.anton, Semrush [Bot] and 140 guests