Host-based backup of VMware vSphere VMs.
craigstrand
Lurker
Posts: 1
Liked: never
Joined: Aug 10, 2010 8:08 pm
Full Name: Craig Strand
Contact:

The object has already been deleted or has not been complete

Post by craigstrand »

" The object has already been deleted or has not been completely created" I get this error when backing up a host that was upgraded from ESX 4.1 to ESXi 5.0. Other posts say to edit the Hosts table directly for the easiest fix, but provide no instructions on how to do so. Anyone have instructions to do this?

Ticket number ID#5210387
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: The object has already been deleted or has not been comp

Post by Vitaliy S. »

Our support team will send you this script after reviewing the debug log files and making sure you are having the known issue. Thanks!
hannisch
Enthusiast
Posts: 34
Liked: 5 times
Joined: Dec 15, 2011 8:14 pm
Full Name: Sven Hannisch
Contact:

Re: The object has already been deleted or has not been comp

Post by hannisch »

Have the same issue, after upgrade from esx 4.1 to esxi 5.0, by doing a fresh install, while retaining the old hostname (only ip changed) I got the same error, by doing a replication job. Regarding this problem I got a ....failed to login to "ESXIHOST5" by SSH error entering the credentials and I have Under infrastructure ->Manged Servers ->VMware vSpere -> vCenter Servers ->VCS-> a duplicate entry of the new host.
Ticket ID: 5213022
ohuetz
Influencer
Posts: 16
Liked: never
Joined: Mar 30, 2011 3:53 pm
Contact:

Re: The object has already been deleted or has not been comp

Post by ohuetz »

Is there any solution to avoid this error? I had it twice with update from 3.5 to 5.0U1 and 4.1 to 5.0U1 both with Veeam Backup 6.1.
hannisch
Enthusiast
Posts: 34
Liked: 5 times
Joined: Dec 15, 2011 8:14 pm
Full Name: Sven Hannisch
Contact:

Re: The object has already been deleted or has not been comp

Post by hannisch » 2 people like this post

Meanwhile I replaced another host. I did this way:

1) Remove host from VCS
2) Rescan VCS in B&R
3) Remove old Host entry from Veeam DB (Hosts Table)
4) Rescan VCS in B&R
5) Install new host and add to VCS
6) Rescan VCS in B&R

This worked for me.
Tobias_Elfstrom
Enthusiast
Posts: 84
Liked: 8 times
Joined: Jul 04, 2012 6:32 am
Full Name: Tobias Elfstrom
Contact:

Re: The object has already been deleted or has not been comp

Post by Tobias_Elfstrom »

Had the same sort of issue when I moved VM's and Hosts to a new vCenter.
Removing the hosts from the dbo.hosts tabled and rescaning within the VBR made the backups work again. Great tip, thanks!
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: The object has already been deleted or has not been comp

Post by dellock6 »

I'm reopening this thread since after a crash of vCenter we had to recreate it...
Long story short, now I'm seeing random errors like this one, not in all our ESXi to be honest. I took a look at the database, is it correct saying the script basically open dbo.hosts table and removed all hosts having type=6 ? Cause I'm seeing also type=3 as proxy/repositories and 1 for Veeam server.
Oh, and there is also a "zombie" repository I removed some time ago, is it something that can happen?

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: The object has already been deleted or has not been comp

Post by Vitaliy S. »

Cannot say for sure, as I didn't see this script, but it's better to consult with our support team before applying any changes to our configuration database manually.
joergr
Veteran
Posts: 391
Liked: 39 times
Joined: Jun 08, 2010 2:01 pm
Full Name: Joerg Riether
Contact:

Re: The object has already been deleted or has not been comp

Post by joergr »

Same as Luca. Had to recreate vCenter Server. It´s not possible to scan the vcenter from within b+r6.5 again unless you manually kill the hosts out of the database. this has to be addressed. The rediscover process should remove orphaned hosts from within an old vcenter where they no longer exist.

Best regards,
Joerg
joergr
Veteran
Posts: 391
Liked: 39 times
Joined: Jun 08, 2010 2:01 pm
Full Name: Joerg Riether
Contact:

Re: The object has already been deleted or has not been comp

Post by joergr »

Wow, now i am thinking positive again - i never thought what crazy sql scripts i am still be able to write after all these years ;-)

But maybe it would be possible to set this as a feature request, just to be sure you can also kill esxi objects when doing a rescan in the database. automatically adding works like a charm, btw - when running a vc rescan missing hosts are written to the hosts db.

Best regards,
Joerg
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: The object has already been deleted or has not been comp

Post by dellock6 »

It would be useful for sure!!
And, another one, being able to decide how frequently we can rescan the vcenter tree for changes. Right now is 24 hours and is fine for the vast majority of customers, but in our environment we change ESXi frequently, and a sysadmin sometimes forgets to run a manual rescan of the hosts, and when DRS kicks in, VMs moving to newly added nodes fails in being backed up.
the error disappears the next run, but in that time we loose a restore point and thus violating our SLAs for customers.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
connelp
Influencer
Posts: 22
Liked: never
Joined: Oct 08, 2012 7:48 am
Contact:

Re: The object has already been deleted or has not been comp

Post by connelp »

Hmmm, I had exactly the same problem last night.

We have three ESXi hosts. I have upgraded 2 from 4.1 to 5 (will upgrade the final one today) and I have come in to find 80% of my jobs have failed.

Just to clarify then, when I have upgraded the final ESXi host should I:

[*]Delete the relevant entries in the dbo.Hosts table
[*]Rescan for my hosts in 'Infrastructure'

Has anyone had to do anything else, like delete and re-add VMs into jobs?

Or should I just raise a call with support?
joergr
Veteran
Posts: 391
Liked: 39 times
Joined: Jun 08, 2010 2:01 pm
Full Name: Joerg Riether
Contact:

Re: The object has already been deleted or has not been comp

Post by joergr »

No, please don´t do that, if you are not highly sql experienced. VEEAM Support will create a sql script for you, if you have an active support contract. Please open a support ticket.

Best regards,
Joerg
connelp
Influencer
Posts: 22
Liked: never
Joined: Oct 08, 2012 7:48 am
Contact:

Re: The object has already been deleted or has not been comp

Post by connelp »

Ah OK, thanks for warning me!

I'm quite confident deleting entries from tables but I don't want to ruin my B+R 6.5 installation.

I'd better upgrade my final host and contact support then.
joergr
Veteran
Posts: 391
Liked: 39 times
Joined: Jun 08, 2010 2:01 pm
Full Name: Joerg Riether
Contact:

Re: The object has already been deleted or has not been comp

Post by joergr »

Yep, that will work ;-)

Best regards,
Joerg
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: The object has already been deleted or has not been comp

Post by dellock6 »

Next time I'll write these kind of stuff, better write at the beginning a huge "UNSUPPORTED" warning :)
Connelp, I'm too with joergr to engage support.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
connelp
Influencer
Posts: 22
Liked: never
Joined: Oct 08, 2012 7:48 am
Contact:

Re: The object has already been deleted or has not been comp

Post by connelp »

Thanks all.

I did raise a call with support but I've found this morning that all my backups have worked OK.

I upgraded my final host yesterday, then did a rescan of my hosts in B+R and it seems to have fixed my issues.

So if anyone is having the same problems a simple rescan may fix it!
virtually-green
Novice
Posts: 9
Liked: 4 times
Joined: Jan 09, 2013 8:34 pm
Full Name: Peter Kuczynski
Contact:

Re: The object has already been deleted or has not been comp

Post by virtually-green »

A customer of mine is have the same issue after patching both Vcenter hosts with the latest patches which came out last week I think.

A case is pending with second level since yesterday.

Is there a working how-to doc on this yet?

A VMware community post advised that removing the VM from inventory and re-adding it resolved that issue, but it was an older power and no specific to Veeam, but to snapshots.
"Re: Trouble reverting snapshots - 'already been deleted or has not been completely created'

I know it is almost a year too late. But, if anyone else is getting same error, the solution that worked for me is to remove the VM from inventory and register it again."
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: The object has already been deleted or has not been comp

Post by Vitaliy S. »

As far as I remember there was some kind of SQL script to fix this issue, so re-registering VMs in the inventory is not required.
virtually-green
Novice
Posts: 9
Liked: 4 times
Joined: Jan 09, 2013 8:34 pm
Full Name: Peter Kuczynski
Contact:

Re: The object has already been deleted or has not been comp

Post by virtually-green »

Unfortunately, after receiving the 1st level callback from Veeam, I was promised a callback by second level, hat never came, Two days later now, I removed the entire Veeam app, reinstalled, used new sql 2008 database, and reconfigured all backups. Took three hours, but everything is working. i feel this issue was caused by a VMware patch, dated about a week ago, at that point all VEEAM backups and replicas stopped working.
I know this ISNT Veems fault, given how fast VMware comes out with patches, in that every patch probably cannot be tested fast enough. I think VMWARe needs to work with VEEAM a bit closer, to release patches and test compatibility issues, before patches are released to poor unsuspecting clods like me, who update their customers, only to suffer a outage later, makes me look good to my customer let me tell you. No, I could not charge my customer.
So, if anyone from VMWARE is reading this, PLEASE, help all us out here, were the ones singing your praises, so do work with VEEAM and other vendors.


Peter
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: The object has already been deleted or has not been comp

Post by Vitaliy S. »

Hi Peter,

Usually, these kind of things (VMware patches) do not brake anything. Can you please tell me what patch have you applied recently, so I could check my labs for this patch presence to compare the results?

As to the response time, then it depends on your support contract. Can you please tell me your support case number, so I could check its status.

Thanks!
johndoe123123
Influencer
Posts: 11
Liked: never
Joined: Jul 30, 2012 3:58 pm
Contact:

Re: The object has already been deleted or has not been comp

Post by johndoe123123 »

Same thing here.

Brand new install of trial on brand new 2008 r2 server install.

Added vcenter server
selected cluster to backup (all vms) to local disk
Started job, got the error message mentioned in the subject.

2008 r2 DC server + 6.5 installed + 6.5 patch 1 installed, before even setting up veeam.

case # 00173261

Deleting and re-adding the server/job had no effect.
tyno81
Influencer
Posts: 12
Liked: never
Joined: Jun 23, 2011 10:55 am
Full Name: Martijn
Contact:

[MERGED] : Veeam 6.5 The object has already been deleted

Post by tyno81 »

Hello Guys,

We are experiencing an issue with one of our jobs since we are using a second vCenter server.
All ESXi Hosts and vm's from one cluster are managed by the new vCenter Server.

In Veeam (6.5, latest patch level), i have added the new vCenter server (Managed Servers)
In the Veeam job i have reselect the cluster in the "virtual machines to backup" tab.
When we are running the job the following error will occur;

2/6/2013 8:17:51 AM :: Error: The object has already been deleted or has not been completely created

So i have copied the back-up data and delete the job and repository.
After that, i have created a new job and new repository, but the error still occur.
Anyone any idea how to fix this?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: The object has already been deleted or has not been comp

Post by veremin »

Hi, Martijn.

Your post has been merged to the existing topic which is related to the same or relatively the same problem.

Please open a ticket with our support team and let them investigate your issue directly, since it seems that the references to the host should be removed from your existing SQL-database manually and our support team will guide you through this process after reviewing attached logs and ascertaining that it’s known error.

Hope this helps.
Thanks.
tomas.olsen
Veeam ProPartner
Posts: 64
Liked: 9 times
Joined: Apr 26, 2011 10:18 pm
Full Name: Tomas Olsen
Contact:

Re: The object has already been deleted or has not been comp

Post by tomas.olsen » 1 person likes this post

Hello, I just had the same issue with vsphere 5.1 build 914609 and veeam B&R 6.5 build 128.
did a complete rescan of the virtual center and revoked all my licenses in veeam. re-ran the job and it worked like a charme :-)
virtually-green
Novice
Posts: 9
Liked: 4 times
Joined: Jan 09, 2013 8:34 pm
Full Name: Peter Kuczynski
Contact:

[MERGED] Case # 00186713 -The object has already

Post by virtually-green »

One of my customers experienced a majour outage on their only vmware hosted server, we had to re-install ESXi and bring back the VM's from the local DS.

This morning I noticed several of their servers are getting the error below, but the latest patch was applied. I even re-applied it this morning.

Code: Select all

3/5/2013 9:35:07 PM :: Error: The object has already been deleted or has not been completely created I have re-applied the patch in this KB 
Veeam version 6.5.0.128

Thanks in advance for the assistance.
virtually-green
Novice
Posts: 9
Liked: 4 times
Joined: Jan 09, 2013 8:34 pm
Full Name: Peter Kuczynski
Contact:

Re: The object has already been deleted or has not been comp

Post by virtually-green » 2 people like this post

This was resolved by Veeam support by editing the esxi host which did not exist any more out of the sql database.
Thanks Veeam!
Peter
SeanD
Influencer
Posts: 16
Liked: 3 times
Joined: Feb 01, 2011 5:33 pm
Full Name: Sean Donaghey
Contact:

[MERGED] : Storage Discovery Error

Post by SeanD »

When the B&R v6.5 server runs a storage discovery I get the following error

Disk and volumes doscovery for VMware vCenter 'vcenter' failed Error. The object has already been deleted or has not been completely created

Any idea on how to resolve this?

Thanks,

Sean Donaghey
travis.vanholland
Influencer
Posts: 13
Liked: 2 times
Joined: May 15, 2012 8:02 pm
Full Name: Travis Van Holland
Contact:

Re: Storage Discovery Error

Post by travis.vanholland »

Check your hosts that are listed in your Veeam infrastructure view. There are probably some in there that you have removed from your VMware environment. As far as I know there is no way to remove those hosts from Veeam except for calling support and having them remove it via a database edit.
chjones
Expert
Posts: 117
Liked: 31 times
Joined: Oct 30, 2012 7:53 pm
Full Name: Chris Jones
Contact:

Re: The object has already been deleted or has not been comp

Post by chjones »

Hi all,

Sorry to dig up an old thread. Just wanted to post my experience with this, with Veeam 7.0 R2 Patch #4. I had the same errors: "Building VM list Error: The object has already been deleted or has not been completely created" in backup jobs, and "Failed to create processing task for VM ####### Error: Child object with ref "host-52041" for object "Datacenters" (group-d1) was not found." for replication jobs.

We have two 7 Host ESX Clusters and yesterday I had to rebuild one of the hosts as the SD Card ESXi was installed to became unbootable. I came in this morning to find I had about an 80% failure of all jobs.

I did a rescan of vCenter, retried all jobs and they're all working.

The rescan of vCenter in the latest versions takes about a minute compared to 30+ minutes in earlier versions. I know it's a waste to rescan vCenter before every single job, but there must be a better way than scanning once a day in the background, some middle ground that helps everyone.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 79 guests