Powershell that detects duplicate backups

[franklin.quijano]

Opened under Support Case # 04328429

Trying to check if there are duplicate backups created.
- One VM having too many backups from different backup jobs.
- Example:
VM100 backed up by Job1, Job6, and Job4
VM253 backed up by Job3, Job5, and Job7
VM369 backed up by Job2, Job8, and Job9

Searching for other methods aside from checking the backups one-by-one.

Hope there could be a powershell script available. Thank you.

[Vitaliy S.]

While you're waiting for help from the PowerShell Community, I wanted to check if you have Veeam ONE (Availability Suite) available in your deployment or not. If you do have it, then this report is exactly what you're after > VMs Backed Up by Multiple Jobs

[krrish1913]

Hi Vitaliy,

We dont have veeam one licenses for my organisation and we are not in potion to buy the veeam one for one report , where comm-vault offering the same kind of report and management asking me to work for same or else change the product at renewal time

[jhoughes]

I have a snippet of code where I am capturing & outputting details of duplicated VMs in jobs/backups, but the main function of the script is to investigate VMs not in backups. It will gather information of VMs based on being added to a job by VMor by tag.

I would need to expand the output for the duplicates to make it more useful, as it currently outputs only a list to text file. Currently, it lists the VM name but doesn't list details about which job or backup and VM ID.

You'll find it here: https://github.com/jhoughes/VeeamON2020 ... ackups.ps1

[franklin.quijano]

Hello, @Vitaliy.

Same as krrish1913, we don't have VeeamONE as of now (although we might consider having one in the future).

Our issue is a temporary, which might not happen again after removing all the duplicate backups.

So a powershell script could help us ease the burden of checking duplicates one-by-one.

Hello, @jhoughes.

Will be checking your script if this is the one I am looking for.

Thank you both.

[vinack]

Hello,
I use PRTG to monitor my infrastructure and made some script to verify the status of my backup.
Let me know if you are interested by this script ...
You could also consider https://kb.paessler.com/en/topic/72030- ... se-manager ...
Sure you could adapt these scripts to your need.
Kind regards - Vincent

[npitacco]

I am not the original poster, but I would be very interested in your scripts.
Nadia

[vinack]

Nadia,
please find my powershell script here below; let me know if you need some help.
You need to install powershell vm tools via "Install-Module -Name VMware.PowerCLI" ...
Kind regards - Vincent

Code: Select all

<#
.Synopsis
	This script connect to an ESXi or a vCenter server and check the last backup date
.Description
	This script connect to an ESXi or a vCenter server and check the last backup date
	vmware-vm-backup-aged.ps1 [[-ComputerName] <String>] [[-address] <String>] [[-UserName] <String>] [[-user] <String>] [[-Password] <String>] [[-IgnoreList] <String>] [[-IgnoreFolders] <String>] [[-OnlyFolders] <String>] [[-Age] <String>] [[-noBackupOn] <String>] [-IgnorePoweredOff] [-UseNoBackupInNotes] [-ignoreNoBackupFolder] [-useNotes] [-v] [<CommonParameters>]
	Where:
	-ComputerName         = vCenter or ESXi IP Address or hostname
	-UserName             = vCenter or ESXi admin account
	-Password             = Password used to connect
	-IgnoreList           = specify machine to ignore
	-IgnoreFolders        = specify folders to ignore
	-OnlyFolders          = check only folder specify
	-Age                  = max backup age in day
	-noBackupOn           = on which day a backup is skyped
	-IgnorePoweredOff     = ignore powered off machine
	-UseNoBackupInNotes   = add NO-BACKUP in notes to ignore machine during test
	-ignoreNoBackupFolder = machine placed in a folder named NO-BACKUP or NO_BACKUP will be ignored
	-useNotes             = use Notes instead of BackupStatus attribute
	-Age                  = reserved for future use
	-v                    = Verbose mode
	Within your VEEAM job add a VM attribute under Backup Job > Storage > Advanced > Notification. Default Attribute is "BackupStatus".
.Notes
	Author:    acv@jumping-net.com
	Version:   20.06.12-00
	Created:   2020-04-24
	Modified:  2020-06-14
	Original Script by https://kb.paessler.com/en/topic/29313-vmware-snapshots
	Modified by Nexpert AG for use with PRTG Network Monitor
	Modified by Jumping NET SA in order to check backup status using VEEAM Notification Tag
.Link
	http://jumping-net.com
    _  _  _  _   _  ___  ____  _  _   ___   _  _  ___  _____   _____    _
   | || || || \ / || _ \|_  _|| \| | / __| | \| || __||_   _| (   __|  /_\
 _ | || |/ ||  V  ||  _/ _||_ |    || (_ | |    || _|   | |    _\ \   / _ \
 \___||____||_| |_||_|  |____||_|\_| \___| |_|\_||___|  |_|   |_____)/_/ \_\
-----------------------------------------------------------------------------
.Example
	vmware-vm-backup-aged.ps1' -ComputerName vcenter01 -UserName administrator@vsphere.local -Password Password -ignoreNoBackupFolder -UseNoBackupInNotes -IgnorePoweredOff -noBackupOn Monday
#>


Param(
[string]$ComputerName = 'vcenter',
[string]$address = '',
[string]$UserName = 'administrator@vsphere.local',
[string]$user = '',
[string]$Password = '',
[string]$IgnoreList = 'FOeFj5gc,asdflkj23C',
[string]$IgnoreFolders = 'asfasf,lkjlj',
[string]$OnlyFolders = '',
[string]$Age = 1,
[string]$noBackupOn = '',
[switch]$IgnorePoweredOff = $False,
[switch]$UseNoBackupInNotes = $False,
[switch]$ignoreNoBackupFolder = $False,
[switch]$useNotes = $False,
[switch]$v = $False
)

Trap {
	Write-Verbose "There is a terminating error: $_"
	write-host "<prtg>"
	write-host "<error>"
	write-host "1"
	write-host "</error>"
	write-host "<text>"
	write-host "There is a terminating error: $_"
	write-host "</text>"
	write-host "</prtg>"
	exit 9
}

if ($address) {
	$ComputerName = $address
}
if ($user) {
	$UserName = $user
}
if ($v) {
	$VerbosePreference = "Continue"
}
#create credentials 
$SecPassword = ConvertTo-SecureString $Password -AsPlainText -Force
$cred = new-object -typename System.Management.Automation.PSCredential ($UserName, $secPassword)

#Import-Module VMware.VimAutomation.Core | Out-Null
if([System.IO.File]::Exists('C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Modules\VMware.VimAutomation.Sdk\VMware.VimAutomation.Sdk.psd1')) {
	Try {
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Modules\VMware.VimAutomation.Sdk\VMware.VimAutomation.Sdk.psd1"
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Modules\VMware.VimAutomation.Common\VMware.VimAutomation.Common.psd1"
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Modules\VMware.VimAutomation.Cis.Core\VMware.VimAutomation.Cis.Core.psd1"
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Modules\VMware.VimAutomation.Core\VMware.VimAutomation.Core.psd1"
	}
	Catch {}
} else {
	if([System.IO.File]::Exists('C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules\VMware.VimAutomation.Sdk\VMware.VimAutomation.Sdk.psd1')) {
#		Add-PSSnapin VMWare.VIMAutomation.core | Out-Null
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules\VMware.VimAutomation.Sdk\VMware.VimAutomation.Sdk.psd1"
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules\VMware.VimAutomation.Cis.Core\VMware.VimAutomation.Cis.Core.psd1"
		Import-Module "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Modules\VMware.VimAutomation.Core\VMware.VimAutomation.Core.psd1"
	} else {
#		Add-PSSnapin VMWare.VIMAutomation.core | Out-Null
<#
		#error
		write-host "<prtg>"
		write-host "<error>"
		write-host "1"
		write-host "</error>"
		write-host "<text>"
		write-host "Initialize-PowerCLIEnvironment.ps1 not available"
		write-host "</text>"
		write-host "</prtg>"
		break
#>
	}
}

Try {	
Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false > $null
} Catch {}

Write-Verbose "Currently connected to $global:defaultviserver"

Write-Verbose "Try to connect to $ComputerName ..."

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls,[System.Net.SecurityProtocolType]::Tls11,[System.Net.SecurityProtocolType]::Tls12

Connect-VIServer -Server $ComputerName -Protocol 'https' -UserName $UserName -Password $Password | Out-Null

Write-Verbose "Currently connected to $global:defaultviserver"

if (! $global:defaultviserver) {
	#error
	write-host "<prtg>"
	write-host "<error>"
	write-host "1"
	write-host "</error>"
	write-host "<text>"
	write-host "Unable to connect to $ComputerName"
	write-host "</text>"
	write-host "</prtg>"
	break
}
$global:textvar = ""

$powerstate = [string]$Args[5]
$old_backup = 0

$IgnoreListSplit = $IgnoreList -Split ","
$IgnoreFoldersSplit = $IgnoreFolders -Split ","
$OnlyFoldersSplit = $OnlyFolders -Split ","

$vmProcessed = 0
$erroneousVm = 0

Function Convert-DateString 
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
        [string] $Date
      )

    BEGIN {
	#Write-Host "Input string: $Date"
    $result = New-Object DateTime
    [string[]]$validFormats = "MM/dd/yyyy hh:mm:ss tt","MM/dd/yyyy hh:mm:ss","MM/dd/yyyy HH:mm:ss","dd/MM/yyyy HH:mm:ss","dd/MM/yyyy hh:mm:ss",
                            "MM.dd.yyyy hh:mm:ss","MM.dd.yyyy HH:mm:ss","dd.MM.yyyy HH:mm:ss","dd.MM.yyyy hh:mm:ss",
                            "M.dd.yyyy HH:mm:ss","M.dd.yyyy hh:mm:ss","dd.M.yyyy HH:mm:ss","dd.M.yyyy hh:mm:ss",
                            "M.dd.yyyy H:mm:ss","M.dd.yyyy h:mm:ss","dd.M.yyyy h:mm:ss","dd.M.yyyy H:mm:ss",
                            "dd-MMM-yyyy hh:mm:ss","dd-MMM-yyyy HH:mm:ss","dd-MMM-yyyy h:mm:ss","dd-MMM-yyyy H:mm:ss",
                            "dd-MMM-yy hh:mm:ss","dd-MMM-yy HH:mm:ss","dd-MMM-yy h:mm:ss","dd-MMM-yy H:mm:ss",
                            "MM-dd-yyyy hh:mm:ss","MM-dd-yyyy HH:mm:ss","dd-MM-yyyy hh:mm:ss","dd-MM-yyyy HH:mm:ss",
                            "MM/dd/yyyy HH:mm:ss","MM.dd.yyyy HH:mm:ss","dd/MM/yyyy HH:mm:ss","dd.MM.yyyy HH:mm:ss",
                            "M/dd/yyyy HH:mm:ss tt","M/dd/yyyy HH:mm:ss","M/dd/yyyy hh:mm:ss tt","M/dd/yyyy hh:mm:ss","dd/M/yyyy hh:mm:ss","dd/M/yyyy HH:mm:ss",
                            "M/dd/yyyy H:mm:ss","M/dd/yyyy h:mm:ss","dd/M/yyyy h:mm:ss","dd/M/yyyy H:mm:ss",
                            "M/d/yyyy HH:mm:ss","M/d/yyyy hh:mm:ss","M/d/yyyy h:mm:ss","M/d/yyyy H:mm:ss",
                            "yyyy.MM.dd HH:mm:ss","yyyy.MM.dd hh:mm:ss",
                            "yyyy.MM.dd H:mm:ss","yyyy.MM.dd h:mm:ss",
                            "yyyy.MM.dd. H:mm:ss","yyyy.MM.dd. h:mm:ss",
                            "yyyy/MM/dd hh:mm:ss","yyyy/MM/dd HH:mm:ss",
                            "yyyy-MM-dd hh:mm:ss","yyyy-MM-dd HH:mm:ss",
                            "ddd MM-dd-yyyy hh:mm:ss","ddd MM-dd-yyyy HH:mm:ss","ddd dd-MM-yyyy hh:mm:ss","ddd dd-MM-yyyy HH:mm:ss"
    }

PROCESS {
    $IsValidDate = [DateTime]::TryParseExact($Date,$validFormats,[System.Globalization.CultureInfo]::InvariantCulture,[System.Globalization.DateTimeStyles]::None,[ref]$result)

    If ($IsValidDate) {
        $result
    } else {
		[datetime]::ParseExact("01.01.1970 00:00:00","dd.MM.yyyy HH:mm:ss",$null)
	}
}

END {}
}

Get-VM -Location $Args[4] | Where {$IgnoreListSplit -notcontains $_.Name -and $IgnoreFoldersSplit -notcontains $_.Folder} | ForEach-Object {
	$vmName = $_.Name
	$vmFolder = $_.Folder
	$vmProcessed = $vmProcessed + 1
	$backupStatus = $_.CustomFields.Values
	if ($useNotes) {
		$backupStatus = $_.Notes
	}
	Write-Verbose "Working on $vmName @ $vmFolder"
	#$vmFolderBackupAge = Get-CustomAttribute -Global -Name "BackupAge" 
	Try {
		$folderCustField = (Get-Folder -Name $vmFolder).CustomFields
		$folderBackupAge = $folderCustField['BackupAge']
	}
	Catch {
		$folderBackupAge = ''
	}
	if ($OnlyFolders -ne "" -and $OnlyFoldersSplit -notcontains $vmFolder) {
		Write-Verbose "		...not in $OnlyFolders, ignored"
	} else {
		$bb = $backupStatus | Out-String
		if ($bb) {
			Try {
				$bb=$bb.Substring($bb.IndexOf("Veeam Backup:"), $bb.Length - $bb.IndexOf("Veeam Backup:"))
			}
			Catch {}
			$b = ($bb -split ",")[1]
			$jobNameString = ($bb -split ",")[0]
		} else {
			$b=($_.Value -split ",")[1]
			$jobNameString = ($_.Value -split ",")[0]
		}
		#Write-Verbose "b = $b"
		if ($_.PowerState -eq "PoweredOff" -and $IgnorePoweredOff) {
			Write-Verbose "		...Powered Off, ignored"
		} else {
			if (($_.Notes -match "NO_BACKUP" -and $UseNoBackupInNotes) -or ($_.Folder -match "NO_BACKUP" -and $ignoreNoBackupFolder) -or ($_.Folder -match "NO-BACKUP" -and $ignoreNoBackupFolder) ) {
				Write-Verbose "		...Noted as NO_BACKUP or located in a NO-BACKUP Folder, ignored"
			} else {
				if ($b) {
					if ($_.Notes -match "Backup-Max-Age") {
						$N = $_.Notes
						$Age4Check = $N.Substring($N.IndexOf("[")+1,$N.IndexOf("]")-$N.IndexOf("[")-1)
					} else {
						if ($folderBackupAge) {
							$Age4Check = $folderBackupAge
						} else {
							$Age4Check = $Age
						}
					}
					Write-Verbose "		...Age limit is $Age4Check"
#					Try {
					if ($jobNameString.IndexOf("[") -gt 1) {
						$jobName=$jobNameString.Substring($jobNameString.IndexOf("[")+1,$jobNameString.IndexOf("]")-$jobNameString.IndexOf("[")-1)
						$c=$b.Substring($b.IndexOf("[")+1,$b.IndexOf("]")-$b.IndexOf("[")-1)
						Write-Verbose $c
						Try {
							$d=[datetime]::ParseExact($c,"dd.MM.yyyy HH:mm:ss",$null)
						}
						Catch {
							$d=[datetime]::ParseExact($c,"M/d/yyyy h:m:s tt",$null)
						}
					} else {
						$jobName = "-"
						$c = ""
						$d = [datetime]::ParseExact("01.01.1970 00:00:00","dd.MM.yyyy HH:mm:ss",$null)
					}
						# $d=Convert-DateString $c
#					} Catch {
#						$jobName = "-"
#						$c = ""
#						$d = [datetime]::ParseExact("01.01.1970 00:00:00","dd.MM.yyyy HH:mm:ss",$null)
#					}
					Write-Verbose "		...Job Name is $jobName, Backup Time is $d"
					$zz = $(Get-Date)
					$yy = $zz - $d
					$xx = $yy.days * 24 + $yy.hours
					Write-Verbose "		...Day Diff between $d and $zz is $yy or $xx hours"
					if ($noBackupOn) {
						if ((get-date).DayOfWeek -match $noBackupOn) {
							$Age4Check = [int]$Age4Check + 1
							Write-Verbose "		...No backup on $noBackupOn, Age limit is now $Age4Check"
						}
					}
					$AgeHours = [int]$Age4Check * 24
					if ( $xx -gt $AgeHours ) {
					Write-Verbose "				...Backup too old, $xx hours"
						$old_backup = $old_backup + 1
						$global:textvar += $_.Name
						$global:textvar += "("
						$global:textvar += $jobName
						$global:textvar += "," 
						$global:textvar += $d
						$global:textvar += "," 
						$global:textvar += $_.PowerState
						$global:textvar += "," 
						$global:textvar += $Age4Check
						if ($Age4Check -eq 1) {
							$global:textvar += " day)"
						} else {
							$global:textvar += " days)"
						}
						$global:textvar += " - "
						$erroneousVm = $erroneousVm + 1
					}
				} else {
					$old_backup = $old_backup + 1
					$global:textvar += $_.Name
					$global:textvar += "("
					$global:textvar += "-"
					$global:textvar += "," 
					$global:textvar += "-"
					$global:textvar += "," 
					$global:textvar += $_.PowerState
					$global:textvar += ")"
					$global:textvar += " - "
					$erroneousVm = $erroneousVm + 1
				}
			}
		}
	}
}

Disconnect-VIServer -Server $ComputerName -Force -Confirm:$False

$x=[string]$old_backup+":"+$global:textvar

#write-host "$old_backup"

if ( $vmProcessed -eq 0 ) {
	#error
	write-host "<prtg>"
	write-host "<error>"
	write-host "1"
	write-host "</error>"
	write-host "<text>"
	write-host "BackupStatus probably not configured"
	write-host "</text>"
	write-host "</prtg>"
	break
}

if($old_backup -ne 0){
	write-host "<prtg>"
	write-host "<error>"
	write-host "1"
	write-host "</error>"
	write-host "<text>"
	write-host $vmProcessed VMs processed and $erroneousVm errors found: $global:textvar
	write-host "</text>"
	write-host "</prtg>"
}

if($old_backup -eq 0){
	write-host "<prtg>"
	write-host "<result>"
	write-host "<channel>Backup Status</channel>"
	write-host "<value>"
	write-host $old_backup
	write-host "</value>"
	write-host "<LimitMaxError>0</LimitMaxError>"
	write-host "<LimitMode>1</LimitMode>"
	write-host "</result>"
	write-host "<text>"
	write-host -NoNewline $vmProcessed VMs processed and no Backup older than $Age4Check days found.
	if ($noBackupOn) {
		write-host Limit changed because no backup set to $noBackupOn
	} else {
		write-host
	}
	write-host "</text>"
	write-host "</prtg>"
}

[npitacco]

Thank you very much.
I will attempt to implement monitoring (I am a PRTG novice)
Nadia