PowerShell script exchange
Shogan

PowerShell - Get-VbrJob

Post by Shogan »

Hi all,

I am looking into the possibility of writing a small report on backup sessions that have completed in Veeam B&R using PowerShell in my spare time. I saw the cmdlet Get-VbrJob has a property "LastBackupSession" which returns the times of the last backup session. My question is, is it possible to rather get a list of backup sessions for the last week for example instead of just the "LastBackupSession"?

This way I can start reporting on things like how long each job took for each day of the week and hopefully further expand on that...

Understood that this may not be a supported topic, but any useful info would help. Thanks!
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Check out Get-VBRBackupSession, just filter as per required


e.g.

Code: Select all

Get-VBRBackupSession | ?{$_.Name -eq "JobName"} | Sort Logger -descending | Select -First 20
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Forget using the Logger for sort as it won't work but if you drill in a little further into the .info you get better detail than the stupid logger anyhoo
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Great! Thank you ThomasMc, I'll have a bit more of a dig around through available cmdlets when I get some time to look at this. Just tried out the Get-VBRBackupSession as you suggested and that has exactly what I was after for a start.

I see what you mean by the logger. Info has just what I need though - I can work with the Start and End times there. :)
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Another question - I started looking at this on my lunch break, and made a bit of progress. Now what I am looking for is a way to retrieve the data processing rate of a past Backup session. Any ideas? Nothing I can see in Get-VBRBackupSession seems useful at the moment. Is there another cmdlet or way to get at this kind of info?
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

I haven't been looking for that value due to it being a bit ambiguous but if you look at the code I posted over here

http://forums.veeam.com/viewtopic.php?f=2&t=9390

That will get you your actual backup size thus giving you start, stop and size so you could calculate your real tf rate
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

if your not sure about comparing dates and times have a look below as PS makes it nice and easy

Code: Select all

$a = "22/11/2011 18:02"
$b = "22/11/2011 18:08"
New-Timespan $a $b
and so get what your looking for its

Code: Select all

(New-Timespan $a $b).TotalSeconds
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Thanks for the follow up ThomasMc :) Yes, I had worked out that subtracting a datetime from a datetime gave me a timespan. What I have done is take the timespan value and format it as 00:00:00 as in hh:mm:ss so that part is working nicely :) Right now I am working on your function from the other thread and finding out how I can match up the current Backup file it is looking at, to the current job session I am iterating through. That way I will tabulate the report based on each job session name, along with all the info I need!

Getting there :)
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Awesome, sounds as if your nearly done, I would love to see your finished script :)
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

I'll be sure to post as soon as its ready. Thanks for your help thus far :)
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Hi ThomasMc,

I got quite close to finishing yesterday, but am struggling to get my script right to match the correct records up (I am using a combination of Get-VBRJob objects to match against a combination of Get-VBRJobSession objects to give me all the data I need). I am using a nested foreach loop.

So for example, I would like to report on the last 3 sessions of each job that exists. So I do this:

Code: Select all

$Report = @()
$SessionsToFetch = 3
$indexvalue = 0
$Jobs = Get-VBRJob
foreach ($job in $Jobs) {
	$indexvalue = 0
	$BackupSessions = Get-VBRBackupSession | ?{$_.Name -eq $job.Name} | sort -Property $_.Info.CreationTime -Descending | Select -First $SessionsToFetch
	foreach ($session in $BackupSessions) {
		
		$row = "" | Select JobName, FullBackup, StartTime, EndTime, Duration, BackupSize, DedupRatio, Compression, Result
		$JobStats = Get-JobStatistics $job.Name
		$indexvalue++ #only increment index value after jobstatistics have been fetched from function as index value of job creation should match backupsession value.
		$row.JobName = $job.Name
		$row.FullBackup = $session.Info.IsStartFullMode
		$row.StartTime = $session.Info.CreationTime
		$row.EndTime = $session.Info.EndTime
		$Duration = $session.Info.EndTime - $session.Info.CreationTime #When subtracting two Date-Time data types you are left with a [TimeSpan] data type.
		$TimeSpan = $Duration #TimeSpan here
		# you would have to get the desired TimeSpan values to the Format operator
		$Duration = '{0:00}:{1:00}:{2:00}' -f $TimeSpan.Hours, $TimeSpan.Minutes, $TimeSpan.Seconds
		$row.Duration = $Duration
		$row.BackupSize = "{0:N3}" -f $JobStats.BackupSize
		$row.DedupRatio = $JobStats.DedupRatio
		$row.Compression = $JobStats.CompressRatio
		$row.Result = $session.Info.Result
		
		$Report += $row
		}
	}
Any way thats not the whole lot, there is a modified version of your function in there that helps gather the data, I just am looking at finding a good way of getting: Job name, start time, end time, duration, backup file size, whether or not it was a full backup, dedup ratio, compression ratio and success/no success all for the same backup session.

The above works, almost. It gathers all this info, but sometimes the wrong job session run is matched against the wrong backup session time. For example I might have one that shows it is a full backup, but the size is clearly an incremental run. Then below that, I see the full backup run size for an incremental run. Any better way of achieving this?
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

I see what you mean, I've been having a little dig about to try and find a relation, I can't figure it out but I did uncover that in the Get-VBRBackupSession if you drill into Info.Progress it gives you a real speed i.e

In the GUI my rate was 2GB / s

and in PS it shows

137085132 for average speed and a quick / 1MB reveals 130.73 MB /s
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Nice find Thomas! I'll definitely add that into the report. If we can't find a work around to relate the jobs with the job sessions, I'll see what else I can figure out today/tonight and hope to soon have my script up here.
Sethbartlett
Veteran
Posts: 282
Liked: 26 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: PowerShell - Get-VbrJob

Post by Sethbartlett »

Shogan, could you show me your example of the incremental/full backup size mismatch you are seeing?
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Sethbartlett wrote:Shogan, could you show me your example of the incremental/full backup size mismatch you are seeing?
Hi Seth,

Sure, here is the output. Note how for the job "Daily VM Backup" there is a true value for the one job session listed there for "FullBackup" - meaning it should be the full backup - i.e. the .VBK. However, filesize is shown as 0.009GB - which is actually the file size for one of the incremental backups listed two rows below that session - and that row lists the correct full backup size of 1.78GB, even though that session is the incremental run.

Here is the output: I'll show you a screenshot of the actual backups of this job so you can match up how it should be too.

Image

Screenshot of "Daily VM Backup" backup files and stats:

Image
Sethbartlett
Veteran
Posts: 282
Liked: 26 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: PowerShell - Get-VbrJob

Post by Sethbartlett »

I have been trying to go over some of this, it looks like it may be something with your code that is flipping them the wrong way. If you notice, it is completely backwards. I notice you are doing

Code: Select all

 $JobStats = Get-JobStatistics $job.Name
Which is probably linking everything together as I would imagine(Doing get-vbrbackup, get-vbrsession and such) but is matching them backwards basically. Does this make sense?
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Is it just me or does the VBRBackupSessions not like sorting properly
Sethbartlett
Veteran
Posts: 282
Liked: 26 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: PowerShell - Get-VbrJob

Post by Sethbartlett »

I agree completely, it doesn't sort by anything from the looks of it >_<
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

this should be enough to get you back on track

Code: Select all

$jobName = "JobName"

$job = Get-VBRJob | ?{$_.Name -eq $jobName}

$table = New-Object system.Data.DataTable "$table01"

$col1 = New-Object system.Data.DataColumn JobName,([string])
$col2 = New-Object system.Data.DataColumn StartTime,([DateTime])
$col3 = New-Object system.Data.DataColumn StopTime,([DateTime])

$table.columns.add($col1)
$table.columns.add($col2)
$table.columns.add($col3)


$session = Get-VBRBackupSession | ?{$_.JobId -eq $job.Id} | %{$row = $table.NewRow();$row.JobName = $_.Info.JobName;$row.StartTime = $_.Info.CreationTime;$row.StopTime = $_.Info.EndTime;$table.Rows.Add($row)}

$intrestingsesh = $table | Sort StartTime -descending | select -first 3

$backup = Get-VBRBackup | ?{$_.JobId -eq $job.Id}

$points = $backup.GetStorages() | sort CreationTime -descending | Select -First 3


$intrestingsesh

$points | Select FileName, CreationTime | ft
[edit]
cleaned it up
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

A better example

Code: Select all

$jobName = "JobName"

$job = Get-VBRJob | ?{$_.Name -eq $jobName}

$table = New-Object system.Data.DataTable "$table01"


$col1 = New-Object system.Data.DataColumn Index,([int])
$col2 = New-Object system.Data.DataColumn JobName,([string])
$col3 = New-Object system.Data.DataColumn StartTime,([DateTime])
$col4 = New-Object system.Data.DataColumn StopTime,([DateTime])
$col5 = New-Object system.Data.DataColumn FileName,([string])
$col6 = New-Object system.Data.DataColumn CreationTime,([DateTime])

$table.columns.add($col1)
$table.columns.add($col2)
$table.columns.add($col3)
$table.columns.add($col4)
$table.columns.add($col5)
$table.columns.add($col6)

$session = Get-VBRBackupSession | ?{$_.JobId -eq $job.Id} | %{$row = $table.NewRow();$row.JobName = $_.Info.JobName;$row.StartTime = $_.Info.CreationTime;$row.StopTime = $_.Info.EndTime;$table.Rows.Add($row)}

$intrestingsesh = $table | Sort StartTime -descending | select -first 3

$pkc = 1

$intrestingsesh | foreach {$_.Index = $pkc; $pkc+=1}

$backup = Get-VBRBackup | ?{$_.JobId -eq $job.Id}

$points = $backup.GetStorages() | sort CreationTime -descending | Select -First 3

$ic = 1
ForEach ($point in $points) {
	$rows = $table | ?{$_.Index -eq $ic}
	ForEach ($row in $rows) {
		($row.FileName = $point.FileName) -and ($row.CreationTime = $point.CreationTime); $ic+=1
	}
}

$intrestingsesh 
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Nice one Thomas.

I just haven't had the time today/tonight to look at this again. I'll have to get to it tomorrow or the weekend :(

Anyway, here is where I got up to. Its generating the info I need, but as I said before, the sorting is not right. Maybe after I look at your examples you have posted above/previously I'll get some ideas to help fix my sorting.

My attempt so far:

Code: Select all

$SessionsToFetch = 3
$indexvalue = 0

Function Get-JobStatistics ($Backup) { 

	$BackupArchive = Get-VBRBackup | ?{$_.JobName -eq $Backup}
	$BackupFiles = $BackupArchive.GetStorages() | sort CreationTime -descending | select -Index $indexvalue
	$BI = $BackupFiles.info
	$BS = $BackupFiles.stats
	[hashtable]$Return = @{} 
	$Return.Name = $Backup
	$Return.FileName = $BackupFiles.FileName
	$Return.BackupSize = $BS.BackupSize / 1GB
	$Return.DedupRatio = $BS.DedupRatio
	$Return.CompressRatio = $BS.CompressRatio
	$Return.CreationTime = $BI.CreationTime.DateTime
	Return $Return 

}

$Report = @()
$Jobs = Get-VBRJob
foreach ($job in $Jobs) {
	$indexvalue = 0
	$BackupSessions = Get-VBRBackupSession | ?{$_.JobId -eq $job.Id} | sort -Property CreationTime -Descending | Select -First $SessionsToFetch
	foreach ($session in $BackupSessions) {
		
		$row = "" | Select JobName, FullBackup, StartTime, EndTime, Duration, BackupSizeGB, DedupRatio, Compression, Result
		$JobStats = Get-JobStatistics $job.Name
		$indexvalue++ #only increment index value after jobstatistics have been fetched from function as index value of job creation should match backupsession value.
		$indexvalue
		$row.JobName = $job.Name
		$row.FullBackup = $session.Info.IsStartFullMode
		$row.StartTime = $session.Info.CreationTime
		$row.EndTime = $session.Info.EndTime
		$Duration = $session.Info.EndTime - $session.Info.CreationTime #When subtracting two Date-Time data types you are left with a [TimeSpan] data type.
		$TimeSpan = $Duration #TimeSpan here
		# you would have to get the desired TimeSpan values to the Format operator
		$Duration = '{0:00}:{1:00}:{2:00}' -f $TimeSpan.Hours, $TimeSpan.Minutes, $TimeSpan.Seconds
		$row.Duration = $Duration
		$row.BackupSizeGB = "{0:N3}" -f $JobStats.BackupSize
		$row.BackupSizeGB
		$row.DedupRatio = $JobStats.DedupRatio
		$row.Compression = $JobStats.CompressRatio
		$row.Result = $session.Info.Result
		
		$Report += $row
		}
	}
Write-Host "Report"
Write-Host "------"
$Report #display report in console
$Report | ConvertTo-Html | Out-File "C:\temp\Veeam_Report2.html" #write report to HTML
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

ThomasMc, I must thank you for your examples and help in this thread :) I have now completed my script with the info I needed via a combination of your code and my own. I have added a couple of fields too and some nice formatting features.

So to summarise, this script runs on your Veeam Backup server/environment, and gathers all the Backup session info for all your jobs (or you can change the -match parameter and search for jobs with certain names/phrases in them to!) It grabs a certain number of sessions you specify for each job, and feeds back all the interesting info to you - such as start time, end time, duration of job, average speed, what kind it was (incremental or full i.e. VBK or VIB), its success/failure etc...

For me, this is great, as I have been tasked with finding out how a large number of jobs are performing (time they complete in and average speeds) (there are 50+ Veeam B&R jobs in our environment)! I can now just run this one script, and gather all the info in one swoop. If anyone wants a modified copy that will email the report to you instead of saving it to c:\ (or would like both) just shout and I'll change it for you to include both.

Lastly, ThomasMc, would you mind if I added this script to my own personal blog? I will attribute help from you there of course :)

Veeam Backup Job / Session statistics and performance report
--------------------------------------------------------------------------
Tested on:
Veeam B&R 5.0.2.230

Requires:
-----------
PowerShell 2.0
Veeam PowerShell module that comes with Veeam B&R 5.0 +

Download here: http://dl.dropbox.com/u/450727/scripts/ ... Report.zip

Example of output - grabbing 3 last sessions for each job on my lab Veeam server (I let the first job fail those 3 times to show how the output changes):
Image
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Awesome work and I don't mind at all :)
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Last update I think lol heres an easy fix

Code: Select all

$VBRBackupSessions = Get-VBRBackupSession
$VBRBackupSessions | Add-Member -type NoteProperty -name StartTime -value ([System.DateTime])
$null = $VBRBackupSessions | %{$_.StartTime = $_.Info.CreationTime}

$VBRBackupSessions | ?{$_.Name -eq "JobName"} | Sort StartTime -Descending | Select -First 3

$ss = $VBRBackupSessions | ?{$_.Name -eq "JobName"} | Sort StartTime -Descending | Select -First 1
$ss | fl
$ss.Info
$ss.progress
mfrycz
Novice
Posts: 4
Liked: never
Joined: Nov 23, 2011 4:05 pm
Full Name: Marcin Frycz
Location: London, UK
Contact:

Re: PowerShell - Get-VbrJob

Post by mfrycz »

Script looks great guys,

I just have few questions:

1. I cannot dig deeper than Get-VBRBackupSession | Get-Member - membertype Property reason for more detail is that I will need to change some results
2. We have several Veeam servers and will need quote against them all
3. Could this be set up to report against particular name (ie. one job can have several vm's and not sure if we could report only on the job level or as grannular as vm level)

I am not very familiar with PS but so far i am amazed by what can it do...

Thanks for all your hard work guys
mfrycz
Novice
Posts: 4
Liked: never
Joined: Nov 23, 2011 4:05 pm
Full Name: Marcin Frycz
Location: London, UK
Contact:

Re: PowerShell - Get-VbrJob

Post by mfrycz »

Hi guys,

this is really good piece of code :)

i am new to ps and trying to amend it slightly:

1. can it report insted of per-job details from per-vm details ?
2. can it specify just particular "*name*" or it have to be across all backups
3. i have several veeam servers, can i quote all of them ?
4. can we add size of the backup ?

thanks again for all your help.
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Not sure is Sean still pops by the forums but you can contact him on twitter(@shogan85) or via his website(Shogan.tech) bearing in mind that your asking for a fair bit of work being done to his original report but since hes a blogger you might just be in luck :D
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

Hey guys,

Thomas, thanks for the mention :)

Marcin, I'll take another look at your questions - possibly later on this week - I have a bunch of exams mid week and as a result have been studying just about flat out in my spare time. I caught the update on this thread in my Veeam Community Digest email this morning :) Regarding your query about looking at stats on other Veeam Backup Servers - I'm sure this can be done using PowerShell Remoting, unfortunately I have only worked a little bit with Remoting - but it should just be a case of connecting to the remote server and running the same cmdlets against it to retrieve data. Anyway, as I mentioned, I'll try look into your queries a bit later this week if I get a chance :)

Sean
ThomasMc
Veteran
Posts: 293
Liked: 19 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: PowerShell - Get-VbrJob

Post by ThomasMc »

Welcome back Sean :) and good luck with the exams.

@Seth I see we got a mention in community mailer :D
Shogan

Re: PowerShell - Get-VbrJob

Post by Shogan »

ThomasMc wrote:Welcome back Sean :) and good luck with the exams.

@Seth I see we got a mention in community mailer :D
Thanks Thomas :) Yes, well done to you guys! I was happy to see a dedicated PowerShell sub-forum pop up a number of weeks back.

PS I also have another useful reporting script to add here soon - as soon as I get a chance to post it up!
Post Reply

Who is online

Users browsing this forum: No registered users and 17 guests