-
- Expert
- Posts: 223
- Liked: 15 times
- Joined: Jul 02, 2009 8:26 pm
- Full Name: Jim
- Contact:
Backups break after host remove/add
Yesterday (for reasons unrelated to veeam) I had to remove one ESX host from vCenter and immediately add it back. Today I see that all of the VMs that were on that host failed their veeam backups, with "object does not exist" errors. I immediately loaded the veeam client and could drill down to my vcenter | host and see everything, so clearly veeam was still able to see the host FINE after it was removed/added back (without needing new/different credentials), but the backups themselves still failed.
I ended up having to edit each backup job to remove the VM and add it right back, after which these broken backups work again. Is this working as designed, or a bug? Obviously I could see this being WAD if a host is missing *at the time* of a backup, but in this case the host was removed and added back to vCenter well before the backups, and veeam itself could clearly browse the "new" host without needing to re-add it to veeam... so why couldn't the backup jobs themselves see the VMs on this "new" host? Some sort of authentication or signature issue? Also (in case it matters), these VMs are chosen from the VM/Template view in veeam, NOT the individual host view.
Mainly I'm just curious if this is the way veeam is supposed to react to this situation, or if it is perhaps a bug.
I ended up having to edit each backup job to remove the VM and add it right back, after which these broken backups work again. Is this working as designed, or a bug? Obviously I could see this being WAD if a host is missing *at the time* of a backup, but in this case the host was removed and added back to vCenter well before the backups, and veeam itself could clearly browse the "new" host without needing to re-add it to veeam... so why couldn't the backup jobs themselves see the VMs on this "new" host? Some sort of authentication or signature issue? Also (in case it matters), these VMs are chosen from the VM/Template view in veeam, NOT the individual host view.
Mainly I'm just curious if this is the way veeam is supposed to react to this situation, or if it is perhaps a bug.
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backups break after host remove/add
Jim, VMs are memorized in the Veeam Backup job using unique IDs assigned to each VM by VirtualCenter (when you first register a VM). After you remove an ESX host from vCenter, vCenter database is cleaned up from all VMs which were present on the removed host. After that, if you add the host back, vCenter does not attempt to match VMs on that host with some historical data, and instead simply registers all VMs present on the host like new VMs, assigning them new unique IDs. And obviously "old" unique IDs memorized in the Veeam Backup settings no longer work.
I recommend setting up the job on the container level (Cluster, VM folder, Datacenter, etc.), instead of by adding individual VMs. This will help to avoid similar issues after operations that result in new VM IDs generation (cloning, re-regisration, and so on).
More generally speaking, managing VMs using this unique ID is what VMware recommendeds vendors to implement, as it is the only way to reliably identify and match the VM (instead of using VM name, IP address and so on).
Hope this helps!
I recommend setting up the job on the container level (Cluster, VM folder, Datacenter, etc.), instead of by adding individual VMs. This will help to avoid similar issues after operations that result in new VM IDs generation (cloning, re-regisration, and so on).
More generally speaking, managing VMs using this unique ID is what VMware recommendeds vendors to implement, as it is the only way to reliably identify and match the VM (instead of using VM name, IP address and so on).
Hope this helps!
-
- Expert
- Posts: 223
- Liked: 15 times
- Joined: Jul 02, 2009 8:26 pm
- Full Name: Jim
- Contact:
Re: Backups break after host remove/add
Thanks Gostev, makes sense. I'm glad veeam takes the cautious approach and doesn't assume a VM with the same name IS the same VM So, you say the better method is to use folders? Would a correct implementation be to put similar VMs (with similar backup needs and schedules, OS type, etc) into folders and then back up at the folder level?
I actually prefer having individual jobs for each VM (the benefits of de-dupe don't help me much) but I could never figure out how to get one job to start immediately after the last, and had to "guess" at the backup window (usually wasting LOTs of time between backups, especially now with CBT!), so I instead chose the indivudual VMs within the same job... but I guess that's not the best, either?
IS there a simple way to "string together" individual backup jobs so that the next starts immediately after the last, and so on?
I actually prefer having individual jobs for each VM (the benefits of de-dupe don't help me much) but I could never figure out how to get one job to start immediately after the last, and had to "guess" at the backup window (usually wasting LOTs of time between backups, especially now with CBT!), so I instead chose the indivudual VMs within the same job... but I guess that's not the best, either?
IS there a simple way to "string together" individual backup jobs so that the next starts immediately after the last, and so on?
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backups break after host remove/add
It takes a little leg work, but you can do it. When you go through a job's properties on the last page is a command line, copy this and paste it into the advanced setting for Post Job Activity. Keep doing this for all your jobs and they will daisy chain like a champ!
h
h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- Expert
- Posts: 223
- Liked: 15 times
- Joined: Jul 02, 2009 8:26 pm
- Full Name: Jim
- Contact:
Re: Backups break after host remove/add
Ah, cool trick! I assume I would then DISABLE the schedule on all but the first job, correct?
-
- Veteran
- Posts: 262
- Liked: never
- Joined: Jul 21, 2009 3:19 pm
- Full Name: Darhl
- Location: Pacific Northwest
- Contact:
Re: Backups break after host remove/add
Exactly!
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backups break after host remove/add
Yes, that is correct. If you are not using VM folders today, this gives you great flexibility to create and stuff them any way you like. I know that most of our customers had been using the VM folders heavily to categorize VMs even before they purchased Veeam Backup, and based on feedback it sounds like many of them decided to use those existing VM folders to organize backup jobs.pufferdude wrote:Thanks Gostev, makes sense. I'm glad veeam takes the cautious approach and doesn't assume a VM with the same name IS the same VM So, you say the better method is to use folders? Would a correct implementation be to put similar VMs (with similar backup needs and schedules, OS type, etc) into folders and then back up at the folder level?
Of course, if the number of VMs is quite limited for you, you can simply continue adding VMs to jobs individually, and rebuild the jobs in cases when you have to re-register VMs.
-
- Veeam ProPartner
- Posts: 76
- Liked: 7 times
- Joined: Apr 14, 2011 3:20 pm
- Full Name: Matthew Ogden
- Contact:
Re: Backups break after host remove/add
I am seeing this same error, I had to add my ESX host manually to backup vSphere properly. in VC, they are added as IPs (and show this was in veeam), so I added them again manually (according to this thread) http://www.veeam.com/forums/viewtopic.p ... 684#p31994
Now, I get the error: Building VMs list
The object has already been deleted or has not been completely created
When I try do a replication job. (New job, I just added). But I am using the VC objects only, why am I seeing this error?
Now, I get the error: Building VMs list
The object has already been deleted or has not been completely created
When I try do a replication job. (New job, I just added). But I am using the VC objects only, why am I seeing this error?
Re: Backups break after host remove/add
Hello Matthew,
For this technical issue, please contact our support team in order to get further assistance. We need to investigate your logs. Thanks!
For this technical issue, please contact our support team in order to get further assistance. We need to investigate your logs. Thanks!
Who is online
Users browsing this forum: Bing [Bot], Semrush [Bot] and 97 guests