Comprehensive data protection for all workloads
Post Reply
Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 13, 2011 8:26 pm

I am in the process of testing out Veeam's replication functionality, and I've come across a problem:

Currently, I have Veeam create a replicated VM from Host A to Host B, over the network. That works fine. If I go into Vcenter and power up the replicated VM, that also works fine (although I was queried initially about the status of the VM - copy/moved/etc.). However, if I attempt to power up the VM using the Vsphere CLI, I get the response "no virtual machine found". I can use the CLI to identify the precise location of the .vmx file, copy and paste that into the command line, and it still doesn't work. My question is - why is the CLI failing to see the virtual machine during the power up command?

Vitaliy S.
Product Manager
Posts: 23432
Liked: 1666 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Vitaliy S. » Apr 13, 2011 8:52 pm

Daniel, can you power up any other VM with the same vSphere CLI command? By the way, what is the command you're using?

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 13, 2011 8:57 pm

Vitaliy S. wrote:Daniel, can you power up any other VM with the same vSphere CLI command? By the way, what is the command you're using?
Absolutely. The command I am using is vmware-cmd.pl (usernamepassword) (host) /vmfs/volumes/vol1/Veeambackup/vm(vm-56)/vm.vmx start

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 15, 2011 1:07 pm

Any ideas here, gents? Does Veeam somehow re-name the .vmx?

Gostev
SVP, Product Management
Posts: 25588
Liked: 3900 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Gostev » Apr 15, 2011 1:23 pm

No, definitely not. We backup and restore VMX file intact.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 15, 2011 3:02 pm

Gostev wrote:No, definitely not. We backup and restore VMX file intact.
What about the configuration file? Why would a CLI command that works for normal VMclones not work for a Veeam clone?

Gostev
SVP, Product Management
Posts: 25588
Liked: 3900 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Gostev » Apr 15, 2011 3:10 pm

I was talking about configuration file above... VMX is VM configuration file.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 2:10 pm

I *believe* I have foudn the problem. Veeam, when making a replication, adds a parenthesis to the storage path: (vm-56) in this case. Apparently the CLI doesn't like that, and I've tried quotes, etc.

Question: can I reconfigure Veeam to not add parenthesis to the file path when it creates a clone?

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 2:16 pm

Have you tried preceding the ( with a \ ? That's typically the escape character in most shells to tell it to interpret the next character as a literal.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 2:18 pm

tsightler wrote:Have you tried preceding the ( with a \? That's typically the escape character in most shells to tell it to interpret the next character as a literal.
Both parenthesis? Like so? \(vm-56\)

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 2:19 pm

Exactly like that, I just tried it on my ESX 4.1 host and that worked fine.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 2:19 pm

Just tried, no joy.

Here is the exact CLI command I am using and the result reported (please note, that file location is a copy paste from the -l output):

C:\Program Files (x86)\VMware\VMware vSphere CLI>vmware-cmd.pl -U ****** -P ******* -H 10.0.0.20 /vmfs/volumes/4daa0249-2482052d-0074-0026b949729a/VeeamBackup/Mail\(vm-228\)/Mail3.vmx start

No virtual machine found.

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 2:25 pm

Are you ESX or ESXi? Is this the remote console?

Like I said I just did exactly this:

vmware-cmd /vmfs/volumes/VMFS_REPL/VeeamBackup/zso-testvm\(vm-29075\)/zso-testvm.vmx start

And it worked fine.

Also, you do know that the "normal" way to fire up a Veeam replica is to use the Veeam GUI, right? Doing it directly through vCenter or the command line breaks the ability to continue replication without starting a full replica again. I'm assuming your testing for the case where Veeam is not available?

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 2:27 pm

Oh, so it is remote CLI. That means you need to use "windows" syntax for escape. Give me a minute, I think I can come up with it.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 2:31 pm

tsightler wrote:Are you ESX or ESXi? Is this the remote console?

Like I said I just did exactly this:

vmware-cmd /vmfs/volumes/VMFS_REPL/VeeamBackup/zso-testvm\(vm-29075\)/zso-testvm.vmx start

And it worked fine.

Also, you do know that the "normal" way to fire up a Veeam replica is to use the Veeam GUI, right? Doing it directly through vCenter or the command line breaks the ability to continue replication without starting a full replica again. I'm assuming your testing for the case where Veeam is not available?
Vsphere Essentials Enterprise Plus for now - ESX. This command is being run from the Vcenter that hosts Veeam (windows box) against the server VCenter resides on.

I do know this is normally run via Veeam, however, I am trying to write a script that will power on a clone automatically when a VM on another machine goes offline.

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 2:38 pm

OK, understood. I think that Windows typically uses the "^" character for escape, so you might try the command as:

C:\Program Files (x86)\VMware\VMware vSphere CLI>vmware-cmd.pl -U ****** -P ******* -H 10.0.0.20 /vmfs/volumes/4daa0249-2482052d-0074-0026b949729a/VeeamBackup/Mail^(vm-228^)/Mail3.vmx start

I don't happen to have a Windows machine handy with VMware remote CLI installed, but I think the above might work.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 2:47 pm

No joy, damn!

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 4:23 pm

Yep, this problem looks like a problem with the vmware-cmd.pl script of the Remote CLI, but at this point I don't think it has anything to do with the parentheses characters. It appears that the routine which parses the $vm_cfg_path variable assumes that the path will always be "<machine directory>/<machine.vmx>", so basically just one path deep. For a Veeam replicated VM there's an extra directory "VeeamBackup/<machine directory>/<machine.vmx>", and the parsing routing is truncating the <machine.vmx> part.

But, in the meantime, I do have a workaround for you. If you pass the "vm_cfg_path" in the format that the "filter" already needs, then the vmware-cmd.pl script doesn't bother to parse it, it just accepts the path exactly as you type it. The "vm_cfg_path" is in the format of:

[VMFS-VOL-NAME] Filepath/machine.vmx

So for your example above the command would be as follows:

vmware-cmd.pl -U ****** -P ******* -H 10.0.0.20 "[VMFS-VOL] VeeamBackup/Mail(vm-228)/Mail3.vmx" start

Note that you should use the actual name of your specific VMFS volume and, with this format, it won't take the UUID, only the volume friendly name (but that should really be easier), and you'll need to use quotes because there's a space between the volume name and the path.

I can probably come up with a patch for vmware-cmd.pl to actually fix this, but the workaround should be good. Give it a try and let me know.

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 5:35 pm

More research and I've found the problem code in the "generate_cfg_path" subroutine in vmware-cmd.pl. The code that returns the path looks like this:

if($ds_flag == 1) {
return ("[" . $dsname . "] " . $patharr[4] . "\/" . $patharr[5]);
}

So it's hard coded to return the datastore name in brackets, and then only two path elements with a "/" in between. Since the replica's have a third path element (the "VeeamBackup" folder) then it doesn't return the full path to the vmx.

The most ghetto fix is probably to just copy vmware-cmd.pl to vmware-cmd-veeam.pl, and change the line above to:

if($ds_flag == 1) {
return ("[" . $dsname . "] " . $patharr[4] . "\/" . $patharr[5] . "\/" . $patharr[6]);
}

Then use this custom vmware-cmd for your Veeam replica scripts. I tested this and it works. It wouldn't be that difficult to hack up a "correct" fix for this, but I'm not sure it's worth it since the workaround I gave you earlier should work fine.

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 5:36 pm

The volume is named Standy Data Store. As such, I am using this command:

C:\Program Files (x86)\VMware\VMware vSphere CLI>vmware-cmd.pl -U XXX -P XXX -H 10.0.0.20 "[STANDBY DATA STORE] VeeamBackup/Mail(vm-228)/Mail3.vmx" start

Unfortunately, no joy. And let me add I very much apprecaite your help here!

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 5:40 pm

tsightler wrote:More research and I've found the problem code in the "generate_cfg_path" subroutine in vmware-cmd.pl. The code that returns the path looks like this:

if($ds_flag == 1) {
return ("[" . $dsname . "] " . $patharr[4] . "\/" . $patharr[5]);
}

So it's hard coded to return the datastore name in brackets, and then only two path elements with a "/" in between. Since the replica's have a third path element (the "VeeamBackup" folder) then it doesn't return the full path to the vmx.

The most ghetto fix is probably to just copy vmware-cmd.pl to vmware-cmd-veeam.pl, and change the line above to:

if($ds_flag == 1) {
return ("[" . $dsname . "] " . $patharr[4] . "\/" . $patharr[5] . "\/" . $patharr[6]);
}

Then use this custom vmware-cmd for your Veeam replica scripts. I tested this and it works. It wouldn't be that difficult to hack up a "correct" fix for this, but I'm not sure it's worth it since the workaround I gave you earlier should work fine.
I am afraid I don't know how to do what you are suggesting. Specifically "copy vmware-cmd.pl to vmware-cmd-veeam.pl, and change the line above to:".

Danielf
Influencer
Posts: 12
Liked: never
Joined: Apr 13, 2011 8:08 pm
Full Name: Daniel Farcus
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Danielf » Apr 22, 2011 5:47 pm

Never mind, I got it, and you are a GOD AMONGST MEN! Thanks mate.

Gostev
SVP, Product Management
Posts: 25588
Liked: 3900 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by Gostev » Apr 22, 2011 9:36 pm

Danielf wrote:you are a GOD AMONGST MEN!
QFE

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 22, 2011 10:30 pm

So I'm curious, did you get the patched script to work, or the workaround by using the different path format. I'm guessing the patched script because I think another bug would keep you from being able to use the initial workaround I provided as it looks like the script would fail to detect your volume name inside brackets because it has spaces. We actually use dashes in our VMFS volume names instead of spaces because of some similar issue in early versions of VMware's tools back in the ESX 2.x days.

tsightler
VP, Product Management
Posts: 5542
Liked: 2361 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam replicated VM will not power on with CLI

Post by tsightler » Apr 23, 2011 2:42 am

Also, since my first "patch" for vmware-cfg.pl was just a ghetto hack to work specifically with Veeam I thought I'd post a more correct patch that won't break the existing functionality and would work no matter the depth of the path. In the original vmware-cmd.pl replace the following line:

Code: Select all

if($ds_flag == 1) {
return ("[" . $dsname . "] " . $patharr[4] . "\/" . $patharr[5]);
}
with the following:

Code: Select all

if($ds_flag == 1) {
   my $vm_real_path = ("[" . $dsname . "] " . $patharr[4] );
   my @path_elements = @patharr[5 .. $#patharr];
   foreach my $element (@path_elements) {
      $vm_real_path = $vm_real_path . "\/" . $element;
   }
   return $vm_real_path;
}
This code will work with VM's in the VeeamBackup folder, and will not break existing functionality.

BTW, it turns out this problem has been going on a long time. When I just did a search to see if there was any chance of submitting my patch to VMware to get them to incorporate the fix, I found the following thread in the VMware Communities:

http://communities.vmware.com/thread/197897

A couple of people reported the same issue (not with Veeam, but with VM's in subfolders) and there's even a patch posted there as well (slightly different than my patch above, but pretty much the exact same concept). This thread is from 2009 so it doesn't look like it's been fixed yet, but I may still try.

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 50 guests