PowerShell script exchange
Post Reply
ThierryF
Expert
Posts: 135
Liked: 35 times
Joined: Mar 31, 2018 10:20 am
Contact:

Get-VBRJobObject : Incorrect/inaccurate Object (EVmware) Location

Post by ThierryF »

Hello,

Busy migrating Vmware VMs to a new VCenter using VBR Replications,
keeping source migrated VMs in a powered off state for DRP in case of
issue for a while, I try to "power-shell scipt" a way to verify the new
replicated vms to be correctly updated in my vbr backup jobs.

By updated, I mean the source VM (old Vcenter) object to be deleted from backup job
and the new VM (new Vcenter) to be added to the same backup job.

To do so, I started with this basic script :

Code: Select all

foreach ($job in get-vbrjob | sort-object -property name) {
	foreach ($jobobj in (Get-VBRJobObject -job $job )) {
		write-host  "JOB : ", $job.name, " - VM : ", $jobobj.name , " - Path : ", $jobobj.location
	}
}
[Moderator: Added code block for readability]

What I notice is the "$jobobj.location" is not correct and inaccurate.

Let me explain what I mean:

When I replicate my VMs, I use a temp ressource pool to store replicas, first to
separate the being migrated VMs from the migrated and runnins ones and
to easier retrieve migrated VMs when updating vbr backup jobs (see later on)

I then create my replication job with selected VMS.
Then run a replication with source VM running, some VMs are quite big in size.
Replicat suffix (_replica by defaut) set to emtpy to keep same names.
When completed (full sync) and ready for migration, I power off source VMs,
re-run replication job (Incr sync) then start replica VMs.
If ok, after a week, I delete my source VMs.

As I deal with a 500+VMs, and for easier vbr backup job update,
I update my backup jobs with VMs stored in temp resource pool, easier to
retrieve VMs.

When migration is ok and backup job updated, the VMs is moved to another ressource pool.
like [ACC/DEV/PROD/....]_[LINUX/vAPP/WINDOWS]

The issue :
When I delete the migrated source VMs from the VBR Backup job and
add the replica ones, Veeam seems to keep the location of the replica VMs
(being VCENTER/DATA_CENTER/CLUSTER/##TEMP_MIGRATION) at update time
and does not update/take care when the VMs are moved to another ressource pool.
Meaning the $jobobj.location is inaccurate when calling Get-VBRJobObject.

If I run "Find-VBRViEntity -VMsAndTemplates", I retrive all of my VMs,
original and replicated ones, but I cannot reply on VM Name only
to be sure my backups are correct (Source and replicat VMs keep original names and
both still exists on the 2 VCenters) and I do not find a quick way using any ID to make
the join ... It's the reason I used $jobobj.location from Get-VBRJobObject.

As a sample, for a given migrated VM :
location from VBRJobObject : VCENTER/DATA_CENTER/CLUSTER/##TEMP_MIGRATION
location from Find-VBRViEntity : VCENTER/DATA_CENTER/CLUSTER/PROD_WINDOWS

As workaround I parse the $jobobj.location, cuting after VCenter Cluster name.
being : VCENTER/DATA_CENTER/CLUSTER

Any idea ? Is this a bug ?
Any advises

Th
david.domask
Veeam Software
Posts: 2820
Liked: 643 times
Joined: Jun 28, 2016 12:12 pm
Contact:

Re: Get-VBRJobObject : Incorrect/inaccurate Object (EVmware) Location

Post by david.domask »

Hi ThierryF,

>When I delete the migrated source VMs from the VBR Backup job and add the replica ones, Veeam seems to keep the location of the replica VM (being VCENTER/DATA_CENTER/CLUSTER/##TEMP_MIGRATION) at update time and does not update/take care when the VMs are moved to another ressource pool. Meaning the $jobobj.location is inaccurate when calling Get-VBRJobObject.

I think I know what's going on here, but I must admit I'm not sure I understood the workflow correctly or the ultimate goal on your script.

1. You mention "I try to "power-shell scipt" a way to verify the new replicated vms to be correctly updated in my vbr backup job", but looks like the script:

- Fetches all backup jobs
- Prints all job object name + locations

Can you explain a bit more what you're understanding from this output and how it helps? Specifically the location property I'm not quite sure I understand what you're trying to confirm or how you determine success/failure.

2. Is it correct that after a VM has been replicated successfully and the final replication is done, you move the VM to a new resource pool and THEN add it to a job? And when running Get-VBRJobObject, the location still shows the temp resource pool?

Your description was useful, I'm just having a bit of trouble following the movement of the replicated VMs and understanding when you're doing the call with Get-VBRJobObject compared to Find-VBRViEntity. The timing and order of operations is a bit important; at first blush, it sounds like Get-VBRJobObject is somehow polling old data, but as I'm not quite clear on your workflow I'm not confident in this.

If you could confirm/elaborate the above, we can try to advise further.
David Domask | Product Management: Principal Analyst
ThierryF
Expert
Posts: 135
Liked: 35 times
Joined: Mar 31, 2018 10:20 am
Contact:

Re: Get-VBRJobObject : Incorrect/inaccurate Object (EVmware) Location

Post by ThierryF »

Hello,

Thanks for your reply.

Re -1 :

When you move 15 VMs, for sample, being dispatched over different VBR Backup jobs,
you may easily forget one in the VBR Job update process.
This results of old VM still being backuped while new one not.

Let's take a sample... Migrating VM1,2 & 3 from OldVC to NewVC ...
VM4 and 5 NOT being migrated yet.
The reason ? If VMs 1-5 are Acc/Dev/Tst and Prod Vms of a given app,
I could migrate Dev/Test in a first step, then Acc and finally Prod inb different
time slot.

VBR Job 1 : VM1(OldVC), VM2(OldVC), VM3(OldVC), VM4(OldVC), VM5(OldVC)

Let's replicate Initial full sync, I have VM1(OldVC), VM2(OldVC), VM3(OldVC) running
and VM1(NewVC), VM2(NewVC), VM3(NewVC) powered off.
At migration time, I shutdown VM1(OldVC), VM2(OldVC), VM3(OldVC), replicate (Incr-sync) and
start migrted VM1(NewVC), VM2(NewVC), VM3(NewVC).

After migration, VM1(OldVC), VM2(OldVC), VM3(OldVC) still exists and are powered off.
VM1(NewVC), VM2(NewVC), VM3(NewVC) are powered on, running.

If I no not modify the VBRJob 1, then VM1(OldVC), VM2(OldVC), VM3(OldVC) will still be backup,
while VM1(NewVC), VM2(NewVC), VM3(NewVC) not.

I then update the VBR Job 1 as VM1(NewVC), VM2(NewVC), VM3(OldVC), VM4(OldVC), VM5(OldVC)
Take care, VM3(OldVC) is incorrect. Still exist in old VC but VM3(NewVC) is not backuped.

The VBR Backup job will be successfull till VM3(OldVC) is deleted from disk.
At that time, an error will pop-up because VM3(OldVC) does not exist anymore and will
realize that VM3(NewVC) is unprotected, with a potential data lose.

So, if I keep VM1(OldVC), VM2(OldVC), VM3(OldVC) for 1 week after migration,
my VM3(NewVC) is unprotected for 1 week. Reason of my cross-check script.

The script I pasted is just an extract. It takes as input a list of VMs being migrated
and it should compare, with collected location output, if location of these VMs to NOT match
old VCenter anymore and match New Vcenter. It will also care if VMs have be deleted
from job and not be added again.

From VBR Job Edit, when selecting Virtual machine and handling long VM list, it is easy
to miss one or select wrong one. Reason why I migrate by set of VMs using temp RP.

My goal is so to run a powershell script that check from veeam configuration
that all of my migrated VMs to be correctly set-up in backup job(s) just after migration.
A kind of cross-check, Errare humanum est. :-)


Re -2 :
Yes, if you move the replica VM to new resource pool and then update the VBR job,
the location from VBR Job Object does match. But if you move the VM from RP,
then VBR Job Object is inaccurate.

I do not directly understand the role of the location field from VBR Job Object
if the value is not reliable...

Futhermore, from this VBR Job Object object, there are NO way to join the object with
an object that can be retrieved from neither Find-ViVBREntity, Find-VBRxxxxxxxxx

Using object name is not reliable as Veeam accepting mutiple VCenter/HyperV/Cloud ...
You may have 2 DataCenters, using 2 different VCenters, having a given VM with identical
VM Name in each VCenter, both VCenter being known into Veeam.
Find-VBRViEntity will gather both VM Object.
In this case, how can you track with VM from which VCenter is backuped ?

The Location field would be the key solution ...

In my case, VMs have coded names :
LXPTEST1 : LX for Linux, P for Prod. should be in RP PROD_LINUX
VWTTEST2 : VW for VM Windows, T for Test. Should be in RP TEST_WINDOWS
VWPTEST3 : VW for VM Windows, P for Prod. Should be in RP PROD_WINDOWS

Location is also to cross-check the VMs to be stored in correct RP
I agree, for this point, I could use Power-CLI script to do the check but
why reinventing wheel when possible not to do it :-)

Thanks for your attention and care,

Warm regards,

Th
david.domask
Veeam Software
Posts: 2820
Liked: 643 times
Joined: Jun 28, 2016 12:12 pm
Contact:

Re: Get-VBRJobObject : Incorrect/inaccurate Object (EVmware) Location

Post by david.domask »

Many thanks for the detailed explanation Thierry, I believe I understand the challenge here and I think this is an "order of operations" error, not a bug.

>Yes, if you move the replica VM to new resource pool and then update the VBR job, the location from VBR Job Object does match. But if you move the VM from RP, then VBR Job Object is inaccurate.

This is expected until the object in the job has been processed next. Apologies, but we'll need to dive into the internals a bit to explain what's going on.

Find-VBRViEntity and Get-VBRJobObject pull their information from two DIFFERENT locations:

Get-VBRJobObject => Read from Veeam Configuration Database to get the latest information about an object
Find-VBRViEntity => Read from Vmware environment (via Veeam Broker service I believe) to get "near-live" results from VMware. (Broker service is more or less a cache for hierarhcy lookups so the data from Find-VBRViEntity might not be live, but I've not confirmed if this cmdlet polls VMware environment directly or not...I'm assuming it's through Broker)

See the difference? The bolded part is why your script is having issues. With Find-VBRViEntity, we get that information directly from VMware/Broker service, so it knows about it.

Job Objects however aren't updated live, only manually adding/removing the workload will update those items or running a backup job should update the path. So I would not rely on the Location property from Job Objects, it's not reliable. You can match a job object to an object returned by Find-VBRViEntity on the Uuid property both will have. Example:

Code: Select all

$job = Get-VBRJob -Name "dd-uuid-job"
$objects = Get-VBRJobObject -Job $job
$objects

Name                      Type       ApproxSize       Location
----                      ----       ----------       --------
dd-tinyvm                 Include    557 B           some.vcenter.local.veeam
dd-tinyvm_example         Include    1 MB             some.vcenter.local.veeam

$objects[1].Object | select id, name, uuid

Id                                   Name              Uuid
--                                   ----              ----
1e6b0f90-fb15-4962-aaa8-5aee2905a7f4 dd-tinyvm_example 4219cf1f-1493-0699-e918-f53037b17603


Find-VBRViEntity -Name "dd-tinyvm_example" | select id, name, uuid


Id                                             Name              Uuid
--                                             ----              ----
575b8d1b-5a1d-4b44-9378-b3d09a5d159d_vm-153724 dd-tinyvm_example 4219cf1f-1493-0699-e918-f53037b17603
Now, for your case, if I got it right you have two VCs, so the simple way to get an authoritative Uuid when Vms on both vCenters have the exact same names is to use the -Server and -Name parameters on Find-VBRViEntity -- If you pass the new vCenter and the name of the VM, you should get the Uuid from it and that will tell you definitively if it's the old or new VM. Similarly, you can use the host name like you've also stumbled on, but Uuid is what I would use to check if a Vmware entity is actually in a job or not.
David Domask | Product Management: Principal Analyst
ThierryF
Expert
Posts: 135
Liked: 35 times
Joined: Mar 31, 2018 10:20 am
Contact:

Re: Get-VBRJobObject : Incorrect/inaccurate Object (EVmware) Location

Post by ThierryF » 1 person likes this post

Hi David,
Thanks for your detailed reply.
You are correct, stating Get-VBRJobObject to be updated at backup execution.
I will adapt my migration steps and give your last suggestion a check ...

Many many thanks for your time.
Wishing you a great day and in advance, a nice week-end.
Cheers

Th
Post Reply

Who is online

Users browsing this forum: Kosk and 3 guests