Discussions specific to the VMware vSphere hypervisor
Post Reply
foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Jun 10, 2015 10:06 am

As mentioned above, the script will not be published but rather used by support engineers (currently, at the higher tiers only). Please ask your case to be escalated.

Shestakov
Veeam Software
Posts: 6836
Liked: 697 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM moved to new vCenter

Post by Shestakov » Jun 10, 2015 10:21 am

The tool (script) was tested by our QA and now available for Veeam users through support team. However, keep in mind that the Support started to help customers with the tool just yesterday. Please treat possible roughness with understanding.

Christopher,
I`ve contacted our support team, your case will be escalated to the engineers helping with moref-ids.
Thanks!

cjack03
Enthusiast
Posts: 29
Liked: 9 times
Joined: Jul 01, 2013 3:26 pm
Full Name: Christopher
Contact:

Re: VM moved to new vCenter

Post by cjack03 » Jun 10, 2015 12:22 pm

@Shestakov.

Many many thanks! I completely understand the stance of the support team, and I agree you can use something that is not completely validated, I was just hoping to make my life a little easier ;-)

I look forward to hearing from the engineers working on this in the future.

Massimo Vitali
Novice
Posts: 4
Liked: never
Joined: Sep 09, 2015 12:43 pm
Full Name: Massimo Vitali
Contact:

[MERGED] Migrating VMware Infrastructure to a new vCenter se

Post by Massimo Vitali » Sep 09, 2015 2:11 pm

We are migrating our environment from vSphere 5.1 to vSphere 6.0.
The first step was to upgrade Veeam Backup and Replication to the latest version and patch level, in order to have full support for vSphere 6.0. Our Veeam server is now at version 8.0.0.2030 and is ready to fully support the new vSphere environment.
Next step was to upgrade Virtual Center from 5.1 to 6.0. We decided to do a fresh installation of Virtual Center 6.0 on a new server instead of doing in-place migration, starting in a new, clean environment. We set up a new server running Windows 2012 R2 and vCenter Server 6.0.0 and replicated the same environment of the old server (clusters, folders, alarms, permissions). The new server obviously has a new IP and DNS name, and we plan to keep both servers active until the whole migration process is completed. We are now ready to move hosts and VMs from the old to the new vCenter server.
Which is the correct way to do this operation at Veeam Server side? We obviously want to keep existing jobs, repositories, restore points, CBT data, etc.
We have 3 clusters in our vSphere environment and, at Veeam side, we have a separate backup job and repository for each cluster. Each backup job includes all the VMs running on the corresponding cluster.
My idea is to move each cluster from the old to the new vCenter server and then modify the corresponding backup job to reflect this change. I plan to move the clusters in separate days, so during the migration period Veaam server must manage some jobs through the old vCenter Server and other jobs through the new vCenter server.

I plan to follow these steps:
  • 1. On Veeam server, add the new vCenter server to Veeam inventory (Backup Infrastructure - Managed Servers)
    2. Unregister all hosts/VMs in Cluster 1 from the old vCenter server and register them on the new vCenter server (obviously when the corresponding Veeam backup job is not running)
    3. On Veeam server, update the registration of hosts/VMs in the backup job to point to the new vCenter server
    4. Wait for the next scheduled job to complete to ensure the backup works correctly
    5. Repeat steps 2-4 for Cluster 2 and Cluster 3
    6. Remove the old vCenter server from Veeam inventory
    7. Turn off old vCenter Server
My questions are:
  • A. Is this the correct procedure to do the operation?
    B. Which are the detailed operations to execute step 3? (update the Veeam job to make it point to the new vCenter Server)
Regarding question B, there are some instructions in KB1299, but my VMs are not directly registered in the job (the whole cluster container is included in the job), so I think I cannot remove and add again each VM separately. I should remove and add again the whole cluster, but I think is not possible to have an empty job for some time. This step is quite obscure for me.
Can you help me giving the correct detailed sequence to succesfully execute step 3?

Thank you very much for your help and best regards

Massimo Vitali

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Sep 09, 2015 2:52 pm

Massimo, in general the procedure looks good, however it omits the important fact that after being registered in the new vCenter all VMs will get new MorefIDs, which will result in Veeam B&R treating them as completely new ones (and backing them up in a full mode). To avoid that, you can contact support for assistance, I believe they will be able to help, if your scenario meets the corresponding requirements.

scott.edwards
Influencer
Posts: 12
Liked: 1 time
Joined: May 01, 2014 5:50 pm
Full Name: Scott Edwards
Contact:

Re: VM moved to new vCenter

Post by scott.edwards » Sep 09, 2015 3:16 pm

Foggy is on target here as I just did the same migration except to vCenter 5.5. To avoid a full backup you should be able to edit a job and map it to the existing chain on your repository.

Massimo Vitali
Novice
Posts: 4
Liked: never
Joined: Sep 09, 2015 12:43 pm
Full Name: Massimo Vitali
Contact:

Re: VM moved to new vCenter

Post by Massimo Vitali » Sep 10, 2015 9:56 am

scott.edwards wrote:Foggy is on target here as I just did the same migration except to vCenter 5.5. To avoid a full backup you should be able to edit a job and map it to the existing chain on your repository.
Thank you to Foggy and Scott for the clarifications. I carefully read all posts in this thread and I understood that I can map the same jobs/repositories to the VMs through the new vCenter, but in any way the first job execution will produce a full backup of each VM because the ref-id of the VMs will change while mapping them through the new vCenter server, so they are seen in Veeam as completely new machines. The only way to avoid this is to instruct Veeam to map the new ref-id's to the old ones using a script under the supervision of Veeam support. Is this correct?

If so, I will open a support request as soon as I will be ready to start the migration, before doing any VM registration on the new server.

Thank you

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Sep 10, 2015 10:10 am

Massimo, you've got everything correctly.

jbsengineer
Enthusiast
Posts: 25
Liked: 3 times
Joined: Nov 10, 2009 2:45 pm
Contact:

Re: VM moved to new vCenter

Post by jbsengineer » Sep 15, 2015 10:02 pm

We are looking at a similar upgrade path as Massimo. Does Veeam recommend we reset the backup job (i.e. Active Full, etc) at anypoint during a 6.0 migration? For example it sounds like we can remap ref-id's and morefs after the vCenter upgrade. But does the changes in CBT on ESXi 6.0 warrant a reset? Or can we seamless upgrade the hosts from 5.1->6.0 and Veeam wouldn't notice a difference. We have around 1 petabyte of Veeam data so resetting it, as you can imagine is difficult.

Thanks!

veremin
Product Manager
Posts: 16710
Liked: 1396 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VM moved to new vCenter

Post by veremin » Sep 16, 2015 10:48 am

It will be a completely new vCenter introduced, right? Not just a in-place upgrade. In this case, once VMs are moved to a new vCenter, stop all existing VB&R jobs and reach our support team; they will apply the migration utility allowing you to bypass full backup cycle after migration. Thanks.

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Sep 16, 2015 10:59 am

There's no need to trigger active full, however, since new vCenter has no CBT data for the migrated VMs, full VM data read will be required during the next job run after migration (only changes will be actually transferred, though).

BoricuaTec
Service Provider
Posts: 30
Liked: 1 time
Joined: Jul 10, 2015 3:19 pm
Full Name: Luis Rodriguez
Contact:

Re: VM moved to new vCenter

Post by BoricuaTec » Sep 29, 2015 3:20 pm

Is there a migration tool in 8.0? I just called into support and they told me since I moved to 6 and did not reuse the DB that I had to manually remove and add all the VMs in their jobs.... Is this correct? There has to be a tool for this.....?

Shestakov
Veeam Software
Posts: 6836
Liked: 697 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM moved to new vCenter

Post by Shestakov » Sep 29, 2015 3:43 pm

Hello Luis,
You are correct, there is an automation tool, but I`m not sure if it was tested for vSphere 6 as well. I`ll check it with our QA team and let you know shortly.
Thanks.

BoricuaTec
Service Provider
Posts: 30
Liked: 1 time
Joined: Jul 10, 2015 3:19 pm
Full Name: Luis Rodriguez
Contact:

Re: VM moved to new vCenter

Post by BoricuaTec » Sep 29, 2015 6:19 pm

Is Veeam looking into making this easy in the future, this can turn out to be a lot of work.

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Sep 30, 2015 10:21 am

The first step to make such migration scenarios easier was to create the support tool mentioned in this thread.

veremin
Product Manager
Posts: 16710
Liked: 1396 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VM moved to new vCenter

Post by veremin » Sep 30, 2015 10:32 am

By the way, vSphere 6.0 has this issue addressed, as it supports cross vCenter vMotion. So, if you're on VB&R v8 U2 and using vSphere 6.0, you should not face the said problem. Thanks.

Shestakov
Veeam Software
Posts: 6836
Liked: 697 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM moved to new vCenter

Post by Shestakov » Oct 01, 2015 2:20 pm

Got an answer from QA team, the tool works for vCenter v6 as well.
Thanks!

rboynton
Enthusiast
Posts: 57
Liked: 7 times
Joined: Jun 25, 2015 12:59 am
Full Name: Rick Boynton
Contact:

[MERGED] Frustrated!

Post by rboynton » Nov 16, 2015 12:42 pm

Case #01118438:

Using Veeam 8. We have been Veeam users for quite a long time. When the VMWare environment is stable (no changes), then Veeam works very well. My issue comes into play when changes to the VMWare environment occur (using VMWare 5.5). Latest example: We had a scheduled power outage, so we shut down each VM, then all the physical hardware. After the power outage, we started everything back up. One host came back up, but VCenter showed no statistics under the Hosts tab. Also, we were unable to power on any VM's running under that host. Had to put that host in maintenance mode, then remove it from the cluster and add it back in. Seems easy enough, right? Everything under VMWare then worked as expected and we could VMotion to that host and power VM's on. Whew! It seems that all the backup jobs for VM's running on that host failed that night. We had to remove each VM from the job, and add it back in. Of course, the first backup was a full. If that was not bad enough, we run backup copy jobs to a DR site. Guess what? New job, new backup copy. Some of those VM's are huge (2+ TB), which causes issues over a very slow WAN so we must copy to an external drive and move to DR. Very frustrating how Veeam seems to be so sensitive! Instead of looking at how a VM is registered in VCenter, why can't Veeam simply match the .vmx file to a job? If this were the case, what happened to us would be a non-issue.

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Nov 16, 2015 2:00 pm

Rick, the behavior is expected in case you re-register host in virtual infrastructure. This resulted in VM unique identifiers (which are assigned by vCenter and used in Veeam B&R to track VMs) being changed and caused Veeam B&R to treat them as new VMs.

rboynton
Enthusiast
Posts: 57
Liked: 7 times
Joined: Jun 25, 2015 12:59 am
Full Name: Rick Boynton
Contact:

Re: VM moved to new vCenter

Post by rboynton » Nov 16, 2015 2:09 pm

Yes, that is what happened...

The issue for me is the recognition system is far too complex. There are too many ways for things to break. Simple is better in almost all cases. Why not have B&R track VM's by VMX name, then once found, find out what host they are running on, then process? If a host is pulled and put back into VCenter, nothing would break. I know this is a radical change from the way it is done now, but the man-hours to rebuild broken backup jobs AND the backup copy jobs is massive. Please have the development team revisit this. Going forward, this only makes sense.

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Nov 16, 2015 2:13 pm

rboynton wrote:Why not have B&R track VM's by VMX name, then once found, find out what host they are running on, then process?
Because VMX name is not unique.

rboynton
Enthusiast
Posts: 57
Liked: 7 times
Joined: Jun 25, 2015 12:59 am
Full Name: Rick Boynton
Contact:

Re: VM moved to new vCenter

Post by rboynton » Nov 16, 2015 2:43 pm

I am not trying to be difficult here. I have never seen a duplicate VMX file. If Veeam is backing up multiple clusters, then perhaps. The solution to that is to simply tie a backup job to a cluster. To create a job, Veeam would then search the datastores attached to that cluster for the desired VMX file. The current solution is prone to breaking, as I have found out more than once. I love Veeam, and have for many years now. I want the product to continue to be successful, but the current way of doing this is very fragile. In the real world, this is problematic to say the least. The development team is smart, so let them find a way to make this work. Please?

Vitaliy S.
Product Manager
Posts: 22777
Liked: 1526 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM moved to new vCenter

Post by Vitaliy S. » Nov 16, 2015 3:49 pm 1 person likes this post

rboynton wrote:To create a job, Veeam would then search the datastores attached to that cluster for the desired VMX file.
Based on my experience when working with large infrastructures (300+ hosts), datastore search tasks can take up to 24 hours. We are using these tasks in Veeam ONE to search through datastores for orphaned files and orphaned VMs, and I wouldn't recommend relying on this process here.

Querying moref ID is a quick API call and is more useful for tracking objects in vSphere infrastructures. Moreover every object in VMware has a moref ID, so using this property for object matching seems more reasonable.

While my explanation doesn't help you at all, and I understand your concern, I would probably recommend to contact our technical team next time you have the same issue and ask for assistance with moref ID changes in our database, so that you could continue running incremental job passes.

dgomes
Influencer
Posts: 20
Liked: 7 times
Joined: Jan 17, 2015 7:16 pm
Full Name: David
Contact:

Re: VM moved to new vCenter

Post by dgomes » Dec 10, 2015 2:35 pm

Is there any way for Veeam partners to be able to do this themselves? We provide the service/support for the customers we sell Veeam to. We know what causes this problem and we know this script can fix it, but having to open a case on behalf of the customers every time this happens is getting pretty old.

Vitaliy S.
Product Manager
Posts: 22777
Liked: 1526 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM moved to new vCenter

Post by Vitaliy S. » Dec 11, 2015 11:47 am

This script is not trivial and can easily mess everything up, that's why we strongly recommend to do that via our support team, but I see your point as well. How often does this situation happen for your clients?

dgomes
Influencer
Posts: 20
Liked: 7 times
Joined: Jan 17, 2015 7:16 pm
Full Name: David
Contact:

Re: VM moved to new vCenter

Post by dgomes » Dec 14, 2015 9:29 pm

For SMB clients not often (2 - 4 hosts), but larger customers who often play around with their hosts/vcenter servers mess things up despite our warnings. Most of the time we tell them just restart their backup chain for affected jobs but it would be nice to have some easier method to fix this. Maybe a separate tool that asks for user input to gather the required variables to run the script correctly? On some partner-only download section, maybe even require the completion of a training video and a quiz to access it if you really need to cover all your bases.

Although I think the ultimate solution to this problem would be the ability to map VMs in a backup. The backup lists the VMs it contains and you match them to the VMs that are live in your environment. When the backup runs the first time after doing this it then checks if what you mapped makes sense.

foggy
Veeam Software
Posts: 18037
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM moved to new vCenter

Post by foggy » Dec 14, 2015 9:55 pm

I doubt they want to mess things up further by directly interacting with Veeam B&R database.

veremin
Product Manager
Posts: 16710
Liked: 1396 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VM moved to new vCenter

Post by veremin » Dec 15, 2015 8:13 am

If they face this issue frequently, may be it's time to update vSphere 6.0 (once it stabilizes a bit) and put into use cross-vCenter migration functionality it has, forgetting about that problem once a for all. Thanks.

JonSD
Novice
Posts: 5
Liked: never
Joined: Aug 05, 2014 2:11 pm
Full Name: Jon Bucud
Location: San Diego, CA
Contact:

Re: VM moved to new vCenter

Post by JonSD » Dec 22, 2015 7:13 pm

Vitaliy S. wrote:This script is not trivial and can easily mess everything up, that's why we strongly recommend to do that via our support team, but I see your point as well. How often does this situation happen for your clients?
We migrated from the VCSA to vCenter Server this week and ran into the mismatched moref ID issue.

I called into support and opened a case, 01244927, and the support engineer I spoke to was not familiar with the script or any procedures related to migrating to a new vCenter.

Vitaliy S.
Product Manager
Posts: 22777
Liked: 1526 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM moved to new vCenter

Post by Vitaliy S. » Dec 22, 2015 7:58 pm

You can ask to escalate your ticket to a higher tier engineer and reference this thread for more details. Thanks!

Post Reply

Who is online

Users browsing this forum: obijay and 24 guests