GFS rotation post job script

[airzyk]

Does anyone have any powershell scripts or batch files that successfully mimics a grandfather-father-son rotation from a single veeam backup job?

What I'd like to achieve is to just have one veeam backup job set to 14 restore points doing forward incrementals and one active full each sunday, and in that job set a post job activity script that will copy the last active full backup file to a different folder called "weekly" and have those files have a retention of 4 weeks. Then from that folder copy the latest weekly file to a folder called "monthly" and have that retention set to 3 months, and so on and so forth for the "yearly" folder. I have read some threads regarding the scripting of rotations but I'm just wondering if anyone has anything directly related to gfs.

[veremin]

From my perspective, there is no need to write some sophisticated PS script, since everything can be done with the following one in conjunction with Windows Scheduler. This script is responsible for finding latest .vbk-file in given location and transferring it to target:

Code: Select all

$SourcePath = "Path to your repository"
$TargetPath = "Path to location to which you’re willing to copy .vbk-files"
$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
Robocopy $SourcePath $TargetPath $LastVBK ;

All you need to do is to schedule three different task responsible for running three different script modifications:

Weekly task that should be run once a week:

Code: Select all

$SourcePath = "Path to your repository"
$TargetPath = "D:\Weekly"
$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
Robocopy $SourcePath $TargetPath $LastVBK ;

Monthly task that should be run once a month:

Code: Select all

$SourcePath = "D:\Weekly"
$TargetPath = "D:\Monthly"
$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
Robocopy $SourcePath $TargetPath $LastVBK;

Yearly task that should be run once a year:

Code: Select all

$SourcePath = "D:\Monthly"
$TargetPath = "D:\Yearly"
$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
Robocopy $SourcePath $TargetPath $LastVBK;

Hope this helps.
Thanks.

[veremin]

In addition, for necessary retention policy you have to write auditorial part, using “if” method.

Below you can find an example how the weekly task with auditorial part should look like:

Code: Select all

$SourcePath = "Path to your repository"
$TargetPath = "D:\Weekly"
$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
Robocopy $SourcePath $TargetPath $LastVBK ;
if ((Get-ChildItem $TargetPath\*.vbk).count -gt 4)
{
    Get-ChildItem $TargetPath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -Last 1 | Remove-Item
} 

Hope this helps.
Thanks.

[airzyk]

EXACTLY what I've been looking for, thank you so much

[tsightler]

If you wanted to get fancy you could change the robocopy to a hardlink instead and save the robocopy time, at least for the monthly and yearly schedule, and even for the weekly schedule if you are using forward incremental backups. This would save a ton of disk space and time, but assumes all the backup files are stored on the same disk volume, which may not be true.

[bgutwein]

Not to be lame but what format do you save the script in? I tried .ps1 and .vbs and they did not seem to work. When I clicked "Run" in the Windows Task Scheduler it just opened up notepad...

[veremin]

In my case it was PowerShell (.ps1) and everything seemed to work propely. Moreover, there previously has been a customer who used abovementioned script in his environment and he didn’t experience any issues either.

Thanks.

[bgutwein]

Got it! I combined the .ps1 script with this: http://blog.pointbeyond.com/2010/04/23/ ... scheduler/ and that got me off and running. Seems kind of hokey that I have to create a bunch of batch and powershell files to get this to work. Maybe a feature for Veeam 7?

[tdewin]

Or you could do all from Powershell. Did not test it completely but if somebody wants to give it a shot

Weekly on saturday on 10:00

Code: Select all

$trigger = New-JobTrigger -At 10:00 -Weekly -DaysOfweek Saturday
Register-ScheduledJob -Name  "Veeam_Weekly" -Trigger $trigger -ScriptBlock {
	$SourcePath = "Path to your repository"
	$TargetPath = "D:\Weekly"
	$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
	Robocopy $SourcePath $TargetPath $LastVBK ;
}

Monthly 1st Sunday of the month at 8:
Weekly we will check on Sunday if the day "number" is lower then 8, it should result in copy first Sunday of the month .

Code: Select all

$trigger = New-JobTrigger -At 8:00 -Weekly -DaysOfweek Sunday
Register-ScheduledJob -Name  "Veeam_Monthly" -Trigger $trigger -ScriptBlock {
	$now = get-date
	if ( $now.day -lt 8 )
	{
		$SourcePath = "D:\Weekly"
		$TargetPath = "D:\Monthly"
		$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
		Robocopy $SourcePath $TargetPath $LastVBK;
	}
}

Yearly 1st Sunday of the month in Month 5 (may) at 13:
Weekly we will check on Sunday if the day "number" is lower then 8 and if the month is 5

Code: Select all

$trigger = New-JobTrigger -At 13:00 -Weekly -DaysOfweek Sunday
Register-ScheduledJob -Name  "Veeam_Yearly" -Trigger $trigger -ScriptBlock {
	$now = get-date
	if ( $now.day -lt 8 -and $now.month -eq 5)
	{
		$SourcePath = "D:\Monthly"
		$TargetPath = "D:\Yearly"
		$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name 
		Robocopy $SourcePath $TargetPath $LastVBK;
	}
}

[KFeggestad]

I have been testing the portion of the weekly script that deletes the oldest file and found that it does not work correctly. I believe that the issue is due to the creation time not being two digit month. I had four backups for weeks in September and October and because in a string of characters 9 comes after 10, it would delete the wrong file. I simply removed "CreationTime" from the command so that it would sort on filename and it appears to work correctly. The Veeam filenames have the date/time in them and the month is 09 so that it sorts correctly. I hope that this helps others who are attempting this.

Keith

[veremin]

Just our of curiosity - wouldn't it be better to leverage backup copy job with GFS mode instead of dealing with customs scripts? Thanks.

[KFeggestad]

I had spoken with Veeam support and they did not recommend that I use the backup copy job for retention purposes. I apologize but I don't recall the exact reason. I believe that it had to do with resources and the overhead that would be involved in creating the backup copy job. It was my feeling that simply scripting to copy from your standard backups would have less or no impact on your VM's, network, etc. Here is what the tech had sent me as a follow-up to our conversation:

Today on our call, we discussed a the use case for the backup copy job. For more information about the backup copy job, this page from our userguide may be interesting:
http://helpcenter.veeam.com/backup/70/v ... _copy.html
http://helpcenter.veeam.com/backup/70/v ... y_job.html

A post-job script leveraging robocopy is usually a preferable solution for making additional copies of backups for longterm GFS archival locally. While we don't offer support for scripting, our forum is a great resource for questions about the many things that can be done with scripts and powershell in Veeam, and gives you the opportunity to ask questions from our developers. Here is a post with some information on how to set up a script that seems appropriate for your use case:

http://forums.veeam.com/powershell-f26/ ... 16382.html

Keith

[foggy]

A post-job script leveraging robocopy is usually a preferable solution for making additional copies of backups for longterm GFS archival locally.

Sounds questionable, especially if taking into account that backup copy job was designed with this scenario in mind. There's a chance, however, that you have some specific requirements that make backup copy not fit your particular case or probably your environment is not capable of handling backup copy synthetic activity. What is your target storage, btw?

[veremin]

Also, can you provide us wit your support ticket number, so that, we can take look? Thanks.

[KFeggestad]

Veeam Support - Case # 00643121

I wonder if the real issue was that I couldn't create a backup copy job that stored the copies in the same repository. I was able to get backup copy to work only when I choose the offsite repository as my destination. Maybe this is a limitation of Veeam that prompted the tech support person to recommend scripting it.

Keith

[foggy]

In fact it is a known limitation of the backup copy job that it does not look for source restore points in the repository that is specified as the target (this is by design). However, you can easily create additional folder on the same storage, add it as a repository to backup console and specify it as a target for backup copy job.

[veremin]

By all means additional folder creation combined with utilization of backup copy job would be much easier option, that usage of custom PS scripts. Thanks.