Identifying VMs by UUID instead of Object_ID

[geofftx]

Just switched to Veeam for backups and love it, but I have a problem. When we test our disaster recovery twice each year we fail all of our VMs over to other hosts at our DR site, run from there for three days, and then fail back. We have to remove all VMs from inventory on the production hosts before doing a SAN handover to the DR site, add them back to inventory on the DR hosts, and then reverse the process when we fail back. The result of all this is movement is the VMs get different object IDs by the time they get back to our production hosts. Consequently, Veeam can no longer identify the VMs so all backups fail and the backup chain is broken.

I've looked at the BObjects table where you store information about the VMs in backup jobs, and you store the UUIDs of the VMs as well as the object_id. Is there any reason you can't use those non-changing UUIDs to identify the VM instead of the ojbect_id, which changes in my scenario?

Any other suggestions to solve my problem without having to rebuild each backup job, deleting and then re-adding each VM, with the resultant broken chain issue?

Thanks,

Geoff

[Gostev]

Hello, the reason why we do not use UUID is because VMware specifically does not recommend using UUID to track VMs, due to the fact that UUIDs are not always guaranteed to be unique. Unfortunately, I do not see how your scenario can be addressed at this time. However, I would recommend setting up jobs using containers - at least, this way you will not have to rebuild the jobs (as when you add every VM individually). Thanks!

[geofftx]

Interesting. How do object_ids get assigned? Looking at the values in the BOjects table it sure looks like this: 564d2237-e1bd-ade8-d1b0-06ecde689075 (UUID) has a far less likely chance of being duplicated than this: vm-49 (object_id)

Just curious about what makes object_ids especially unique, and why they change when you remove and add a VM back into inventory?

Good tip about the containers... I'll have to make that change.

[Gostev]

UUID itself is designed for, and guaranteed to be unique. However, withing VMware, certain actions can result in to different VMs having the same UUID - this is where the problem is. I cannot say exactly what are those actions (at least from the top of my head), but VMware had it well explained in some development recommendations document that I read a couple of years ago.

[tsightler]

Object ID's are really MoRef (Managed Object Reference) IDs. Everything think in a vcenter has one, folders, resource pools, datastores, etc. They are guaranteed to be unique "per management host", which, in real terms, generally means "per-vCenter" (an standalone ESXi host could technically also be a management host). The combination of the vCenter ID and their MoRef ID will be unique. When you remove the object from vCenter, and the readd a VM, vCenter assigns the object a "new" MoRef ID.

I'm curious about your procedure, why are you require to remove all of the old VMs from inventory to perform a failover? That seems unusual.

[geofftx]

Thanks for the additional info.

Per VMware KB article KB2004605, it's necessary in order to cleanly detach a LUN before you unpresent it from the SAN. During a SAN handover from production to DR, the LUNs are taken offline from production hosts and are then presented to the DR hosts. And as I found out the hard way, ESX really, really doesn't like this happening before you do a detach, especially prior to 5.1. APD (All Paths Down) leads to some extremely unpleasant outcomes, like all your hosts losing communication with vCenter and needing a power cycle to come back up.

[tsightler]

I guess I was assuming that you were failing over the entire environment so you could just power off the hosts at the primary site.

[geofftx]

Good thought, and in a real disaster that's probably what would happen, but we also run a VDI environment on some of those same hosts (connected to a different SAN) that we don't take down during this part of our failover testing, so I need to keep hosts running and cleanly detach LUNs from the SAN that's part of this test. If only I could afford SRM...

[chrisdearden]

You could always look at replicating via Veeam instead - then you could manage the failover through that ( and save lots of cash on san replication licencing )

[geofftx]

A couple of reasons... "Instant" VM recovery with 30+ servers isn't going to work in any practical way, and the bigger issue is failback. How are you going to do that with Veeam (again, in any reasonable period of time)? Don't get me wrong, I love what Veeam does, but my SAN is far better suited for this task. Just wish Veeam had a better way to identify the VMs in a backup set, or gave me a choice to take my chances on the UUID instead of the object_id.

[chrisdearden]

When you use replication you dont do instant recovery - you effectively just power the replica VM on your DR site on Failback is supported - essentially the replication is done in reverse - depending on the level of change during that time , failback might take a little bit longer than if the SAN had already started replicating back. That said if you are happy with your current solution then a move to the container based setup should work for you.

[geofftx]

But doesn't this require the VMDKs are served up from whatever storage you're using for backups, all presented as an NFS connection through the backup server? Not sure performance would be anywhere close to acceptable if that's the case.

[chrisdearden]

No it doesn't - when you back up , the result is a VBK/VRB/VIB file with multiple Vm's data on it - that can be used to instant restore a machine as you suggested. When you create a replication job , the "target" of the replica is an ESX host or cluster. A powered off Virtual machine is created on that target cluster , sitting natively on what ever storage is present. You can keep a number of replication points , just in case you dont want to fail over to the latest replica.

There are a few live demo's and webinars on veeam.com , including one recorded just a few days ago http://www.veeam.com/videos/building-bi ... -1712.html that should help illustrate what I mean
you can also check out the content on Veeam university. http://www.veeam.com/university/failover.swf

[geofftx]

Great, thanks. Looks like I need to spend more time looking at the way Veeam does replication.

Quick licensing question. In order to recover VMs at the DR site, do the procs there need to be licensed, i.e. do all my servers in both the production and DR site need licenses to use the recovery functionality?

[Vitaliy S.]

No, only source ESX(i) hosts should be licensed. Thanks!