18/08/2013 6:43:05 PM :: Failed to create processing task for VM MyVMName Error: Object reference not set to an instance of an object.
Case: 00430381
Pertinent Details:
Error occurs across all replication jobs - all using HotAdd mode. All backups jobs are running ok.
Error occurs across jobs for VM's within the same VCenter and across different VCenters
Have fully recreated one of the jobs - still receive same error
Have reentered credentials for VC and Guest jobs
Proxies were upgraded during install
Proxies in questions also happen to have full VBR console installed at the other end (to do local jobs)
All components have been restarted, and connectivity between all sites tested
No other changes in network or infrastructure at all
Couldn't see anything in the release notes or KB that appear related, except for hitting 'recalculate' to refresh the cache.
All jobs are using seeds and disk exclusions which I noticed weren't bought across correctly from the upgrade - ie all disks were selected after the upgrade (only selected disks were included prior), however fixing this doesn't rectify the error.
Drilling into the job logs doesn't seem to show much from a cursory look - the above error just appears out of nowhere.
I remember something sort of similar during a previous upgrade that required revoking and reapplying host licenses but all seems to be ok on this occasion.
There is a confirmed small bug with replication and VMs with excluded disks. This may be your issue.
Case # 00429720
paraphrased from tech support:
"We seem to have a small release v7 bug with replications with excluded disks. If you wish to workaround this issue while R&D determines next steps please add thin provisioned disks on your target replicas matching the size of the source VMDKs and map your existing replication job to the replica VM with the empty extra disks We will not fill these disks with data, but it will allow you to replicate in the meantime."
I made the changes to my replica VMs, and replication is working again.
Well - it worked for the first run after adding the disks, however the job removed those disks after the successful run, so it failed on the next run..
So this means we effectively have to re-add the disks prior to every job run, to get the failing replication jobs to work?
Replica jobs don't have the 'Remove excluded disks from VM configuration' option like backup jobs do, which could have been handy.
I've been given a fix for this issue (new veeam.backup.core.dll). I haven't tested it yet (waiting for a replication to finish). Nor have I tried a 2nd replication after the 1st worked.
Also note -- this failure can be triggered if you replicate your backup proxy VMs and use hotadd. The foreign disks will cause the replication to fail if it tries to replicate the proxy VM while it's running a job.
we've only upgraded one customer so far, there are 2 replication jobs (for the same VM) that exclude one of its disks. The job seems to be working fine.
What are the causes for disk exclusion not to work?
(we have another site that is using it quite heavily, so we are not sure if its going to work or not)
The first fix didn't quite work with a standalone vmware host as a destination. A 2nd fix from today so far is working.
odge -- Initial v.7 relase code replications simply fail any VM that has an excluded disk (replications only, not backups). Other VMs (even in the same job) can work just fine (at least in reference to this specific issue).
My educated speculation leads me to believe that the process to check for an accurate replication destination VM (whether a seed or an existing replicated VM) runs before checking for excluded disks. Therefore the job is looking for the excluded disk on the replica VM, which (obviously) does not exist, and subsequently fails thinking that it's not actually a useable replica of the VM it's trying to process. I suspect that Veeam just needed to calculate the replica VMs configuration before comparing to verify a valid destination.
Bunce -- I can confirm the workaround of adding thin disks to the destination replica VM will only work one time. Of course, you can add them again after that and get another run to go. (Or get the fix from tech support.)
there must be one other variable, because I'm looking at my v7 job, and its run more than once with
2013/08/20 11:42:58 PM :: Releasing guest
2013/08/20 11:43:29 PM :: Disk '[Perf-iSCSI-3] db-server/db-server.vmdk' has been skipped because it was excluded from processing by user
2013/08/20 11:43:43 PM :: Processing configuration
2013/08/21 11:25:10 PM :: Releasing guest
2013/08/21 11:25:38 PM :: Disk '[Perf-iSCSI-3] db-server/db-server.vmdk' has been skipped because it was excluded from processing by user
2013/08/21 11:25:44 PM :: Preparing replica VM
Eventually gets to success (i see as well, that the output wasn't quite the same, maybe tonight is going to fail?
Fix looks to have worked on some jobs but not others for us. Difference between them looks to be the failed jobs use destinations in a different VCenter.
I'm not sure if we were given the first or second patch though, so will follow up.
I am still doing the trial for veeam 7 and I have an issue after deleting the a job then recreating it utilising the current replica as a seed. It comes up with the error "Task failed Error: Object reference not set to an instance of an object." when I try run the new job. I went down the process of deleting the snapshots off the job and still had issues ultimately when this happened the last time. (These are my sql server replications and I am excluding the backup drive and apparently this is a known bug with veeam ver 7).
Since I am trialing the software, do I have the right to raise a ticket for support on this?
Or does anyone have a patch I could try for 7 to see if it helps?
Hey guys, as thought, and for all those interested, it was caused by jobs that do not replicate all disks. The veeam team gave me the updated dll required to fix this and voila it's resolved. So yeah a bug in the current version.
Thanks for the update, much appreciated. Yes, there is a known issue where replica mapping could fail if the VM has some disks excluded. Currently support has a private fix available for that and the issue will be also fixed in the upcoming patch. I'm now going to merge this thread to an older existing discussion regarding that problem. Thanks.
I have a case open on this exact issue (Case #00447045), the tech assigned didn't mention the private fix as an available option. Any chance you could take a look at this case and see if that would be an option? As of right now I'm told the only workaround is to abandon the seed and run a full replication without the disk exclusion, two things I'd like to avoid unless absolutely necessary.
From you guys and gals receiving the hot fix DLL, did any of you test their guest replica after?
I advice you to do so.
I had a huge surprise, not sure if the v7 upgrade or the hot fix is at fault, but it used to work before.
Replicated win2k3, Disk 0:0 having 3 partitions leaving WSUS repository Disk 0:1 out of it.
The replica seems to contain only partition 0 and 1, leaving the windows install partition (number 2) out of it and hence not booting up. It may be a combination of my setup with the P2V guest and veeam, but I urge everyone to test it out.
Lesson learned! Upgrade or pathch: test after like it would be a new product. Veeam cannot replicate all the possible scenarios in the field.