Host-based backup of VMware vSphere VMs.
Post Reply
th83
Influencer
Posts: 11
Liked: never
Joined: Oct 24, 2011 7:42 pm
Full Name: Tim Haner
Contact:

Quick Migrate ate my VM!

Post by th83 »

Definitely be sure you have a good backup before using quick migrate to move a VM to a new datastore without Storage vMotion. I did this (fortunately with a test VM) and the results were nothing short of terrifying.

The process started and created a 'new' VM, then started copying the data from the old VM to the new one. Once the copy process finished, the old VM was paused, active state migrated (supposedly), then it got scary. The 'old' VM was deleted entirely, put through a paper shredder, destroyed BEFORE the 'new' VM was powered on or tested. The 'new' VM (now with the same name as the old one) would not power on and gave an 'OS Not Found' error. When I looked at the new copied vmdk files, they were both under 1GB, when the original files were about 30-40GB each, so it did not work. Mounting the migrated vmdks showed they had no data and were not formatted. I will not be even attempting to use this feature again as that was about they most frightening thing I've ever seen done to a VM... not testing & verifying the copied VM before shredding the old one is a HUGE, HUUUUUUGE, MONSTROUS, EPIC (you get the picture) design flaw. Love the backup/replication part, but the new quick migrate feature needs some work to say the least. No way I'll use this part in production yet.
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Quick Migrate ate my VM!

Post by Gostev »

Sounds like something has deleted working snapshot before it got applied. Is this reproducible? Hopefully, you will be able to forward the logs to our support for investigation. I've played a lot with this feature and never had anything like this happen. Nevertheless, probably it's good idea not to delete "old" VM automatically (but just unregister it). Or at least delay its removal until heartbeat from "new" VM appears. I will talk to devs about adding this in the next update.
th83
Influencer
Posts: 11
Liked: never
Joined: Oct 24, 2011 7:42 pm
Full Name: Tim Haner
Contact:

Re: Quick Migrate ate my VM!

Post by th83 »

I haven't had the time to re-create the issue yet. Since it was a test VM, I may have had a snapshot on it, so not sure if that is part of it. When I have some free time I'll try again with a freshly cloned test VM. I would like to see some kind of verification in place so that it can automatically fail back to the original VM if the move process doesn't work right, that would make me more comfortable using this feature in production.
marcottt
Enthusiast
Posts: 40
Liked: never
Joined: Sep 08, 2011 2:40 pm
Full Name: Marco Tonoli
Location: Genova
Contact:

Re: Quick Migrate ate my VM!

Post by marcottt »

Hi,
some days ago i try new v6 in my lab, so after install i'll do some little test.
I found quick migrate, so i try this feature with a powered on vm windows xp (host is a esxi5 old ibm 6gb ram with 4 das datastore from scsi storage direct, without vcenter) from a datastore to another datastore of same scsi host.
I'll receive same result of th83, vm do some snapshot, deleted from and recreated new but..... no boot up. I did't check the problem, i was thinking i need to read better manual and the host is very old; i deleted all and restored from backup. I think need to study something before new test. Not a problem now.
But now i read the th83 post.... so i thinks is strange.
I think the BIG problem is that old vm was deleted before test.... this is strange from veeam... surebackup, vm tools heartbeat, ping, anything but please do some test before delete vm... rename it meanwhile o some similar...

Not a problem for me, was a xp test in lab.... but if happen in production.....

Bye
Marco
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Quick Migrate ate my VM!

Post by Gostev »

I hope either of you will be able to forward the logs to our support, since we cannot reproduce this behavior of migration failing in such a way. As for the heartbeat test, it will be added in the next update. Thanks!
iowasolutions
Lurker
Posts: 1
Liked: never
Joined: Mar 04, 2012 4:04 pm
Full Name: Iowa Solutions
Contact:

Re: Quick Migrate ate my VM!

Post by iowasolutions »

I had the exact same problem the very first time I tried a quick migration as well.

OS NOT FOUND
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Quick Migrate ate my VM!

Post by foggy »

It would be much appreciated if you could open the case and provide logs for deep investigation.
Mathieu
Novice
Posts: 6
Liked: never
Joined: Sep 07, 2011 2:31 pm
Full Name: Mathieu
Contact:

Re: Quick Migrate ate my VM!

Post by Mathieu »

Today I have installed veeam b&r 6.1 at our test server. I also was currious about the quick migration because we don't have the storage vmotion option from vmware (license) but I also have the same result as the topic starter...

I have the idea that the new vm is using the same vmdk as the original disk (through snapshot?) and so the migrated boots fine (and the wait for heartbeat function also works fine), after that the source/original vm is deleted and the new/mrigated vm crashes and rebots with the "OS not found" error.

I will post some screenshots later on and will also open a support case for this issue. Will also retry this with the option to delete the source checked out.
Mathieu
Novice
Posts: 6
Liked: never
Joined: Sep 07, 2011 2:31 pm
Full Name: Mathieu
Contact:

Re: Quick Migrate ate my VM!

Post by Mathieu »

I have done the same test again (with a new VM because the other one was deleted) and have now done 3 successfull quick migrations (moving the VM from one datastore to another).
Below I have posted two screenshots, the first is from the failed migration, the second from a successfull one. In the first you can see that no data is moved at the step "Copying VM files". Also a warning symbol is displayed but no reason for that is given. Is there some low level logging available in veeam to see why the first test failed?

Image

Image
Vitaliy S.
VP, Product Management
Posts: 27377
Liked: 2800 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Quick Migrate ate my VM!

Post by Vitaliy S. »

Mathieu wrote:Is there some low level logging available in veeam to see why the first test failed?
Yes, to browse to the log files, select Help > Support Information from the main menu.
LenChaney
Novice
Posts: 6
Liked: 1 time
Joined: Dec 06, 2012 9:13 pm
Full Name: Len Chaney
Contact:

Re: Quick Migrate ate my VM!

Post by LenChaney » 1 person likes this post

I have been having a similar issue with QuickMigrate - which is why I'm here today.

My issue is that the process stops at the "Waiting for Heartbeat" message. I may stay there for 10 minutes or an hour. Normally, it takes a few minutes and will go on and complete the QuickMigrate.

If the process hangs on "Waiting for Heartbeat", you can delete the VM with the original name and after a few minutes, Veeam will detect this and restore everything to its original state. i.e It will rename the "_migrated" VM to the original name, reconfigure the VM, fix the snapshots, and reconfigure the Backup Proxy and end the job with an error.

The QuickMigrate will work the second time. I'm not entirely sure why, bit it will. I' trying to get a pattern for this and figure out how to avoid it.

I would like to find a fix so I can queue up a bunch of quick migrations and go out and grab lunch.

BTW, I have VMWare Essentials with does not have vMotion or Storage vMotion licenses which is why I'm using QuickMIgrates,

Also, with QuickMigration, you can migrate a running server and the only issue is that it gets suspended for a few minutes. This is OK for many of the VMs I am moving to new storage and much cheaper than moving up from VMWare Essentials.

When it works, it works wonderfully. When it does not, it still works OK.
veremin
Product Manager
Posts: 20413
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Quick Migrate ate my VM!

Post by veremin »

Also, with QuickMigration, you can migrate a running server and the only issue is that it gets suspended for a few minutes.
Yep, it’s just a difference between the used approaches. Whenever it’s possible VB&R relies on VMware vMotion mechanism. Otherwise, if for some reason vMotion can’t be used (for instance, If you don’t have appropriate VMware vSphere or you’re willing to migrate VMs between standalone ESX(i) hosts, etc), Veeam Backup & Replication leverages its proprietary SmartSwitch technology, which does have a little downtime.

Thanks.
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Quick Migrate ate my VM!

Post by foggy »

LenChaney wrote:I would like to find a fix so I can queue up a bunch of quick migrations and go out and grab lunch.
Len, if you want to investigate the reason of unsuccessful migrations, I encourage you to open a case with our technical support. Probably logs will tell why do the migrated VM cannot start properly after the first migration attempt.
Post Reply

Who is online

Users browsing this forum: No registered users and 11 guests