HOWTO: Backing up vCenter host issues

PowerShell script exchange

HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Fri Nov 11, 2011 11:03 pm

I'm going to cover something that has been covered before, but with a little twist, an UNSUPPORTED twist. (Which is mostly my powershell stuff :) ) When backing up your vCenter(Virtual Center Server) and trying to use VSS(Guest-Processing in Veeam), you will run into an issue where you will get "Freeze operation timed out, wait 900 seconds". The issue is actually the database being frozen for the Virtual Center server and it cannot update the database/Veeam because it's in a frozen state, so it times out.

The work around for this is to add your ESX/ESXi host into Veeam aside from your VC. So for example:

vSphere Client to the VC Shows:

esx1.host.local
esx2.host.local
esx3.host.local

You would want to add in the host that manages your VC by IP address into Veeam.

vSphere Client to the VC Shows:

192.168.1.100
192.168.1.101
192.168.1.102

You would want to add in the host that manages your VC by netname. If it does not have a netname, you can add an entry into your hosts file on the Veeam box. After you add your host into Veeam, you will need to backup your VC by picking your host directly, rather than drilling down through your VC. The issue that comes with this is that if you vMotion your VC or have to move/re-add to inventory, the reference ID of the VC VM changes. This will in turn, break the job and you will get something like "Object reference not set to an instance of an object" or "Object not found", something along these lines.

So the real question is, how do you fix this without doing a brand new full every time.

The requirements:

- vPower CLI
- Connection to Veeam Database

So the first part, vPower CLI:

Code: Select all
#First, we need to connect to the VC to get information about the host that manages it
#User/Password are optional and a prompt will come up for you to type credentials, you can also supply

PSCredential
$VC = Connect-VIServer "VC" -User "MyUser" -Password "MyPassword"
$myHost = (Get-VM "vCenter Name" -Server $VC).VMHost
if($myHost.name -match "((\d{1,3})\.?){4}")
{
   $ServerName = [System.Net.DNS]::GetHostbyAddress(matches[0])
}
else
{
   $ServerName = [System.Net.DNS]::GetHostAddresses($myHost.name)[0].ipaddresstostring
}

# Now we need to connect to the host that the VC is on
$tempHost = Connect-VIServer $myHost.name -user "MyUser" -Password "MyPassword"
$vm = Get-VM "vCenter Name" -Server $tempHost
$id = $vm.id -replace '[a-z]+-'

asnp "VeeamPSSnapIn"
$Server = Get-VBRServer | ?{$_.name -eq $ServerName }


So from here, we have two choices on how we want to update the DB(You may have more if you so choose)

The script we are going to run is:

Code: Select all
UPDATE [BObjects] set host_id = $Server.id, object_id=$id WHERE object_name = $vm.name AND object_id NOT

LIKE 'vm%'


You can use 'sqlcmd' and the syntax would be:
Code: Select all
sqlcmd -s SERVERNAME\INSTANCE -d Database -q "The query above"


For powershell, you would do the following:
Code: Select all
$Connection = New-Object System.Data.SQLClient.SQLConnection
$Connection.ConnectionString = "server=servername;database=VeeamBackup;trusted_connection=true;"
$Connection.Open()
$Command = New-Object System.Data.SQLClient.SQLCommand
$Command.Connection = $Connection
$Command.CommandText = "UPDATE [BObjects] set host_id = $Server.id, object_id=$id WHERE object_name =

$vm.name AND object_id NOT LIKE 'vm%'"
$Command.ExecuteNonQuery()


This will update your job to the correct host and update the reference ID. Your job will continue change block tracking and will continue like nothing is broke :)
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Fri Nov 11, 2011 11:15 pm

Great work Seth, you where spot on about killing my script :D
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Sat Nov 12, 2011 5:46 pm

Would this get you what you need without the PowerCLI stuff?

Code: Select all
$VCVMName = "vcenter.lab.local"
$esxihost = Get-VBRServer | ?{$_.Type -eq "ESX" -or "ESXi" -and $_.Reference -eq "ha-host" }

foreach ( $esxiserver in $esxihost ) {
   $foundvc = Find-VBRObject -Server $esxiserver | ?{$_.Name -eq $VCVMName } -WarningAction SilentlyContinue
   if ( $foundvc -ne $null ) {
      $newvchost = $esxiserver.RealName
   }
}

$Server = Get-VBRServer | ?{$_.name -eq $newvchost }
$id = $foundvc.Ref
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Sun Nov 13, 2011 8:07 am

That actually works PERFECT! I prefer that method and never even thought of doing it that way. You would need to do $id = $foundvc.Ref inside your foreach loop after setting $newvchost but that would definitely work and provide the exact same results without the need for Power-CLI and some extra DNS crap I am doing!
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Sun Nov 13, 2011 1:11 pm

A little tweak to the -or as well, seems I got that bit wrong lol

Code: Select all
?{($_.Type -eq "ESX") -or ($_.Type -eq "ESXi") -and $_.Reference -eq "ha-host" }


or even this

Code: Select all
?{($_.Type -eq "ESX") -or ($_.Type -eq "ESXi") -and $_.Reference -notlike "host*" }


Been testing this most of last night and its working like a dream, hats off to you for coming up with this :)
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Sun Nov 13, 2011 4:40 pm

add a little zip to it :D

Code: Select all
asnp "VeeamPSSnapIn" -ErrorAction SilentlyContinue

$VCVMName = "vcenter01.lab.local"
$serverName = "$env:computername\VEEAM"           
$databaseName = "VeeamBackup"


$esxihost = Get-VBRServer | ?{($_.Type -eq "ESX") -or ($_.Type -eq "ESXi") -and $_.Reference -notlike "host*"}

foreach ($esxiserver in $esxihost) {
   $foundvc = Find-VBRObject -Server $esxiserver | ?{$_.Name -eq $VCVMName}
   if ( $foundvc -ne $null ) {
   $newvchost = $esxiserver.RealName
   $id = $foundvc.Ref
   }
}

$Server = Get-VBRServer | ?{$_.name -eq $newvchost}
$hostguid = $Server.id.guid
         
$dataAdapter = New-Object System.Data.SqlClient.SqlDataAdapter           
$query = "SELECT * from [BObjects] WHERE object_name = '$VCVMName' AND object_id NOT LIKE 'vm%'"           
$connString = "Server=$serverName;Database=$databaseName;Integrated Security=SSPI;"           
$dataAdapter.SelectCommand = new-object System.Data.SqlClient.SqlCommand ($query,$connString)           
$commandBuilder = new-object System.Data.SqlClient.SqlCommandBuilder $dataAdapter           
$dt = New-Object System.Data.DataTable           

try
{
   $dataAdapter.fill($dt)
}
catch
{
   Write-Warning "Error: Could not fill the Datatable"
   break
}

if ($dt.Row.Count -gt 1) {
   Write-Warning "This might fail with duplicate rows error, old data probably exist"
}

$dt  | foreach {($_.host_id = "$hostguid") -and ($_.object_id = "$id") }

try
{
   $dataAdapter.Update($dt)
}
catch
{
   Write-Warning "Error: Could not update the table"
   break
}
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Sun Nov 13, 2011 11:48 pm

Nicely done. Now you dont have to constantly re-create jobs and such :P
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Mon Nov 14, 2011 10:36 am

Yeah the way I was doing it was a bit clunky :D plus the addition benefits of having CBT and No Re-seeding

10 vMos, 10 Backups, 10 SB and 10 Reps done and not one failed
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Mon Nov 14, 2011 12:58 pm

Awesome! Thanks for fully putting this through the works. I tested it with a few jobs and few vMotions and it looked pretty solid but it looks like you really beat on it :)
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby daryanx » Fri Nov 18, 2011 12:52 am

Huge thanks should go to ThomasMC and Sethbartlett on this. The script has solved a major issue for a lot of people!! Thank you
daryanx
Influencer
 
Posts: 10
Liked: never
Joined: Wed Jun 01, 2011 5:02 am
Full Name: Paul Hutton

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Fri Nov 18, 2011 12:54 am

Any time:) You can actually use this same method to fix any VM issue with the object being removed. So if you had to remove a VM from inventory and re-add it back, you can follow the same type of information in the script and DB update to fix the job.
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Fri Nov 18, 2011 12:06 pm

Glad your finding it useful, I was merely Pinky and Seth was the Brain on this one :D could I ask how your triggering it though, are you manually running it? Scheduled Task? or something else
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Fri Nov 18, 2011 1:09 pm

Well you could do a scheduled task. Another option would be to have it fire off on the vMotion event in VMWare.
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby ThomasMc » Fri Nov 18, 2011 5:13 pm

Sethbartlett wrote:Another option would be to have it fire off on the vMotion event in VMWare.


That's how I'm doing it now, I found 2 ways, one really insecure way(with PsExec firing the script) and another secure way(via encryption and ps remoting). I've got a unpublished blog post that shows you how to use the later with some customisation to the script above but wasn't sure on the etiquette for posting your work on there lol
ThomasMc
Expert
 
Posts: 293
Liked: 17 times
Joined: Wed Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell

Re: HOWTO: Backing up vCenter host issues

Veeam Logoby Sethbartlett » Fri Nov 18, 2011 6:20 pm

You can always re-post any of my stuff :P I label it as unsupported for a reason, since I'm also a support guy :)
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Sethbartlett
Expert
 
Posts: 282
Liked: 25 times
Joined: Wed Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett

Next

Return to PowerShell



Who is online

Users browsing this forum: No registered users and 7 guests