Question about items that are no longer processed by a job and deleted item retention

[jcofin13]

We have some veeam instances where the source vms have been removed or purged from vmware.

Over time if you look in veeam you can still see these items under Disk. The retention in some cases dwindles down to 1 over time and it seems to keep the last full backup. IN other cases it has the last full vbk and a few vibs and yet in other cases i can see all the fulls and all the vibs back 30+ restore points.

What determines what is kept around if the vms is no longer processed by the job and there is NO deleted item retention enabled or set?


Also....am i correct in that to clean all these up the proper way is to set the deleted item retention to enabled and set it for XX number of days (14 by default) and it will delete any and all data on the repo under "disk" that is outside of that number (all vbks and vibs)? I know by default the deleted item retention job setting is not enabled so im just trying to understand the behavior.

It is my understanding that turning on this deleted item retention will NOT clean up any GFS items on the Capacity Tier regardless if your deleted item retention date meaning it will keep all weekly, monthly, yearly etc as defined by the job? Is this accurate?

[david.domask]

Hi jcofin13,

1. What determines what is kept around if the vms is no longer processed by the job and there is NO deleted item retention enabled or set?

Old VMs are only cleared out by retention if they are attached to a job. Background Retention operates on backups in the *Orphaned nodes (e.g., Backups > Disk (Orphaned) ), and operates as described on the linked page.

2. Also....am i correct in that to clean all these up the proper way is to set the deleted item retention to enabled and set it for XX number of days (14 by default) and it will delete any and all data on the repo under "disk" that is outside of that number (all vbks and vibs)?

Correct, it will follow the behavior outlined here in the User Guide.

3. It is my understanding that turning on this deleted item retention will NOT clean up any GFS items on the Capacity Tier regardless if your deleted item retention date meaning it will keep all weekly, monthly, yearly etc as defined by the job? Is this accurate?

Capacity Tier is an extension of Performance Tier backups -- that is, if retention removes a backup on performance tier, it's copies on capacity tier are also included as they do not have separate retention. This is explained in more detail here

If you have a need to keep specific copies for longer, consider Export Backup and target the Scale-out Backup Repository for the export, as these will be copied/moved to capacity tier as with normal backups, and can follow their own retention as well.

[jcofin13]

Thank you for the clarity.

This helps a lot. It must be that the vm was purged prior to the full retention period running out which might explain why we still have more than "1" restore point left on some vms.
Thats about all i can think that would cause that.


Im not sure what you mean by your last sentence about needing to keep copies longer? Backup copies? Longer than what? The deleted item retention?
When exporting it looks like you must select vms individually and individual restore points as well. Are you saying we should set the export targets for individual vm exports to the sobr and it would then place them on the capacity tier and not be tied to a job and thus not be subject to deleted item retention as it wasnt a backed up vm to begin with (it was an export)?

Along those lines it would be nice to be able to export entire jobs to a point in time. As an example, id like to run a "yearly" backup for compliance but i dont want to keep that stuff in the capacity tier (public cloud) just to avoid the storage fees. No reason to pay month over month just to store data thats likely never going to be accessed. Id like to run that yearly job to a local storage and keep it there.
The only way i can think to do this is clone my jobs and change the destination to another sobr or single device with no capacity tier. Exporting entire jobs from a common recovery point (rather than having to choose export per vm would be another way to do this to avoid the "cloned jobs" that would only be used once per year in this example. If you could export an entire job from say the 12/31 backup job that runs and then target that full export to a local storage, it would retain its encryption key(at the time).

[david.domask]

You're very welcome for the answers jcofin13, and glad I could add some clarity. I agree on the first point, likely just retention had "hit" at the right time, I would not put too much thought onto this one unless you're missing backups that were essential, it seems fairly normal situation for me.

> keep copies longer

Mostly just wanted to point out we have options to keep specific out-of-band copies if there is a need before you clear out the original backups. For example, you might not need an entire 20 point chain, but you need one point for archival purposes. Export Backup can do this by making a standalone full backup from any given restore point, and the standalone full backup can have its own retention.

You can do this similarly with the entire backup itself with Copy Backup, which I think is what you're asking in your final paragraph -- you can point that to whatever storage you like (SOBR or non-SOBR target), and avoid the cloning of jobs, shuffling repositories, etc.

[jcofin13]

>"You can do this similarly with the entire backup itself with Copy Backup, "

Ah....duh. There i go overthinking again. Yeah that makes perfect sense and is much more simple.

Thanks again David.

[jcofin13]

Is there a simple report that you can pull with VeeamONE that shows you these vms and their "backed up" size? I have tried the "orphaned vms" report but that comes up blank. The only way i have been able to find the sizes is to look at the disk-->job-->properties and scroll through the massive list there to find the backup size of each one but it doesnt tell you which are orphaned. You have to figure that out first. I just want to find out how much space i can expect to save by turning the deleted item retention on per job.

[david.domask]

Hi jcofin13,

Just to confirm, your goal here is to understand the total size of the backups for VMs still added to jobs but are no longer present in the production environment, correct?

I'm not aware of a report that tracks this specifically, and while possible with Powershell, it would be a bit tricky to calculate as the best value I can think to reliable check it with would be the UUID value on CJobObject and compare it to the results of Find-VBRViEntity -VMsAndTemplates which have the same UUID value, with some additional code to then get backups related to any missing UUIDs.

If I get some time I will try to punch out some pseudo code for it, but may be a bit.

[jcofin13]

Yes.

The vms in this case were added to the jobs individually...not by storage lun, folder or any other kind of grouping.
Once the vms were purged from the environment, the veeam job no longer had the vm to process but left some number of vbk and vib files behind. I suspect this is some number that correlates with the last full and any number of vibs up to the point when the vm was removed. I guess im not 100% sure since some vms only show 1 restore point and others show more.

All I'm looking for is the vms on storage that are no longer processed by a job (and their size) so i can calculate how much the space savings will be if we turn on deleted item retention (thus cleaning up and removing all those left over items). I thought this might be the orphanded vms report in VeeamOne but that one comes up blank when i run it.

I guess i could just enable the deleted item retention, let the jobs run, and then id know
In doing so, id like to try and estimate how much it will clear up on the performance tier as well as the capacity tier. IF i can figure out how much it will clear up on the capacity tier, i will be able to figure out approx. how much will be saved per month.

[david.domask]

Got it, thanks for the information.

I punched out the following script yesterday. Please note a few limitations atm on it:

1. Script will only work if you added VMs individually to jobs -- containers (Resource Pool, Tags, Folders, Hosts, etc) require a different approach which I can add if someone finds this script useful
2. Currently works for VMware backups only
3. Script returns an array intended to be exported to CSV for further review, includes the jobID, backup file name, and file size of the backup file

Save the following code as a .ps1 file, copy it to the VBR server. Open a Veeam Powershell console from the console Menu > Console > Powershell. cd to the script location, and executing it, saving the output to a variable. Example, assuming script is at C:\temp\dvmrreport.ps1:

cd C:\temp
$deletedmachinesizereport = C:\temp\dvmrreport.ps1

Code: Select all

$allJob = Get-VBRJob
$allObjects = @()
Foreach($j in $allJob){
	$obj = Get-VBRJobObject -Job $j
	$objects = $obj.Object | Where-Object {$_.Type -eq "VM" -and $_.ViType -eq "VirtualMachine"}
	If(-not($objects)){Continue}
	$jobID = $j.id
	$data = [PSCustomObject]@{
		JobID = $jobID
		ObjectData = $objects
	}
	$allObjects += $data
}
$entityUUIDs = $allObjects.ObjectData.UUID
$viUUIds = Find-VBRViEntity -VMsAndTemplates | Where-Object {$_.Type -eq "VM"} | Select id, reference, name, uuid
$deletedVMUUIDs = $EntityUUIDs | Where-Object {$_ -notin $viUUIds.Uuid} | Sort -Unique
$deletedVMs = $allObjects | Where-Object {$_.ObjectData.UUID -in $deletedVMUUIds}
$deletedVMBackupStats = @()
Foreach($dvm in $deletedVMs){
	$backup = Get-VBRBackup | Where-Object {$_.JobId -eq $dvm.JobId}
	$rps = Get-VBRRestorePoint -Backup $backup | Where-Object {$_.ObjectID -eq $dvm.ObjectData.id}
	$storages = $rps.GetChainStorages()
	$storageData = @()
	Foreach($s in $storages){
		$data = [PSCustomObject]@{
			JobID = $dvm.jobID
			FileName = $s.PartialPath
			FileSize = $s.Stats.BackupSize
			}
		$storageData += $data
		}
	$deletedVMBackupStats += $storageData
}
	
return $deletedVMBackupStats

[jcofin13]

Thank you David. I will try this out when i get time.

Much appreciated.