Backup to Tape only last backup with reversed incremental

[foggy]

Is it then possible to start a Veeam tape job with powershell?

I believe you can use Start-VBRJob cmdlet to do that (as in the script provided above).

[veremin]

Ok thanks for the answear. Is it then possible to start a Veeam tape job with powershell?

Yep, the script should be pretty simple.

First, you get the corresponding tape job. Then, you start it.

Code: Select all

Get-VBRTapeJob -name "Name of your Tape Job" | Start-VBRJob 

Thanks.

[veremin]

As much as I appreciate this script and the effort by the team it doesn't help us poor SOBs that are using a network repository to store our backups as hardlinks are not supported.

Then, you can automate slightly different approach. The one described above might do the trick for you.

Thanks

[Stuzoo72]

Currently when we use "Tape Job - Backups" and select a backup job it places all the restore points on to tape.

For jobs with a large retention policy this can be a lot of restore points and a lot of duplicated data. This results in many tapes.

I would like Veeam to add a couple of options for "tape jobs - Backups".
Option 1: Place most recent Full backup to tape only. This would place the most recent full backup to tape, whether this be a reverse incremental, active full or old active full with increments.

Option 2: Place most recent Active Full backup to tape.

These options would allow only the data we require for a single point in time backup to be placed on tape and so reduce the costs associated with having to use tape.

Thanks

[veremin]

Hi, Stuart,

Yep, you're right. For now there is no functionality that allows you to copy only the latest full backup to tape. So, one of the options you have is to use the file to tape job, as well, hardlinks. The way this process can be automated is described in this post.

Thanks.

[sbbots]

Otherwise, you can create a folder called “Latest VBK” and specify it as a source for “file to tape” job. Within a script you should find the latest .vbk file, copy it to the aforesaid folder, execute corresponding tape job and delete .vbk file once the job is done. Thanks.

This is exactly what I do and it works perfectly. I use a simple BAT file in Windows Task Scheduler to accomplish this.

Create a new "Files-to-Tape" job with no schedule pointing to your "Latest VBK" folder and copy the job ID at the end of the setup. Next, copy the commands below to a text file, change the RED variables to your setup (including the job ID) and save as a BAT file. Lastly, create a task in Windows Task Scheduler to run the BAT file at the time/date of your choosing. If you want to delete the files after the tape job is complete just put a delete command at the end, but ***make sure you set a delay long enough to allow the tape job to finish before deleting the files***. In my setup I choose to manually delete the files myself.


::Delete all files in directory
del /F /Q "F:\Latest VBK\*.*"

::Copy latest VBK to directory
@echo off
echo. |date |find "current" >> "F:\Latest VBK\Log.txt"
echo. |time |find "current" >> "F:\Latest VBK\Log.txt"
set "source=F:\Backups_Daily\VMHOST01"
set "dest=F:\Latest VBK"
pushd "%source%" ||(echo.Source does not exist&pause&goto EOF)
for /f "tokens=*" %%f in ('dir /A-D /OD /B *.vbk') Do set "file=%%f"
popd
xcopy /d /i "%source%\%file%" "%dest%\" >> "F:\Latest VBK\Log.txt"

::Copy latest Veeam Config to directory
@echo off
set "source=F:\Backups_Daily\VeeamConfigBackup\VMHOST01"
set "dest=F:\Latest VBK"
pushd "%source%" ||(echo.Source does not exist&pause&goto EOF)
for /f "tokens=*" %%f in ('dir /A-D /OD /B *.bco') Do set "file=%%f"
popd
xcopy /d /i "%source%\%file%" "%dest%\" >> "F:\Latest VBK\Log.txt"
echo. |time |find "current" >> "F:\Latest VBK\Log.txt"

::Run Veeam backup job
"C:\Program Files\Veeam\Backup and Replication\Backup\Veeam.Backup.Manager.exe" backup c2d493aa-e310-4f6d-a15c-88d1163e67dc

[mongie]

I would also like an option to only copy the latest full backup to tape, and not all vbk's in the backup chain since the older vbk is already on an older tape. And space and time are often rare on tapes.

I too am waiting for this functionality...

example:

We keen 5 points of retention on all our backup jobs which run daily, monday to friday (and use archive jobs to keep archive retention).

We run MONTHLY backup to tape jobs for all our backups.

With 5 retention points set on the jobs, we normally have 2 full backups on disk at any one time (Forward Incremental - Dedupe unit). Because we only run the tape job monthly, each time it runs, both full backups are new to the tape job, and it writes both. Some of these backups are 2TB+ and this takes too long to write to tape.

[Dima P.]

Alex,
Thank you for providing the valuable use case! We added this request to tape feature list for the upcoming versions.

[chadtandy]

Hi all, we're on Veeam v7r2 and are going to soon be moving from copying Veeam files to tape with Commvault to having Veeam copy directly to tape itself. I've poked around with the scheduling a bit but seem to be missing something. I just want to copy to tape once a week, and I only want to copy the latest full, not the entire backup chain. How do I schedule this?

Thanks, and sorry for my ignorance...

-Chad

[veremin]

Hi, Chad. Your post has been merged into existing discussion.

Currently, there is no such functionality, so, you have to use workarounds (scripts) mentioned above in order to get what you're after. However, we consider adding this functionality in one of the next product releases.

Thanks.

[smallsheep]

I can add my name to the list of those waiting for this functionality. I have a customer who does reverse incremental with backup to tape and they were having instances of two full *.vbk files being backed up in a night which caused the job to spill over onto 2 tapes, they had a standalone drive rather than an auto-changer so this was an issue for them. If it was possible to limit Veeam to backup only the latest full it would be very useful indeed.

[techguy615]

I would like to see this as well. I am toying around with a simple way to do this myself. I am trying to eliminate tapes entirely from the process. I currently have 4-4TB drives attached to a hot swap hard drive bay. I am in the process of migrating my file shares to a VM in order to finally give Symantec the boot. So as for now I am attempting to use good old robocopy to do this.

(Assuming E:\ is the local destination)

Batch File Contents:
------------------------
del E:\*.* /q
robocopy <VeeamRepository#1> E:\ *.vbk /MAXAGE:1 /LOG:C:\RobocopyLogs\Backup.log (Copy files from the Main repository to the local E:\ drive with the vbk extension, excluded ANY files that are older than one day).

------------------------
This way I get ONLY the newest .vbk file written to my local E:\ drive and can take it offsite. I have added the batch file to my "Post Job Activity" in the main backup job. I figure from there someone could copy it to tape if needed or possibly incorporate this method to get the files to tape somehow. This is all just theory though. I am about to run a backup and see what happens. I am hoping it will work without too much trouble. I will post back with the results.

[veremin]

Hi, Chad, haven't you tried to use backup copy job, instead of Robocopy? We have introduced recently some code improvements that allow backup copy job to work with rotated medias; might be worth taking a look.

Thanks.

[techguy615]

I have been playing around with it a bit. I am thinking if I tell it to only keep one restore point it should get the latest vbk file only and not the incremental backup files. Robocopy isn't a preferred way to do any of this of course. I was just looking for a workaround to get something to disk for off site. Everything else seems to be working fine as far as I can tell.

[veremin]

I am thinking if I tell it to only keep one restore point it should get the latest vbk file only and not the incremental backup files.

First of all, the backup copy job doesn't t copy backup file as a whole, but rather synthetically creates required restore points in "target" location from VM data in source backup repositories. Also, with the backup copy job the least number of restore points that can be set is 2. So, with the backup copy job you will always have at least 2 restore points (1 .vbk, 1 .vib). However, with the referenced regkey, backup copy job should work perfectly with rotated media.

So, it's certainly worth trying.

Thanks.

[techguy615]

I have reconfigured my job to use the reverse incremental. I did see where the job was limited to only letting me select 2 as the number of restore points. I have also noticed if I entered a 1 manually instead of clicking the up/down arrows it doesn't complain about it though. That might not be a good thing though. As for the registry entry you are referencing, I am apparently missing it. Nothing like showing up late for the game.

Thanks for all of the help thus far. I am very new to the product at this point.

[veremin]

As for the registry entry you are referencing, I am apparently missing it. Nothing like showing up late for the game.

No problem. This is the registry key I was talking about. This regkey was designed specifically for rotated media scenario, and, once implemented, it will enable automatic cleanup of the backup repository from all existing files if any backup file from the latest full backup chain is missing. So, it might be worth taking a look.

Thanks.

[techguy615]

Sounds good. I downloaded the patch and added the reg keys. Will see what happens.

[veremin]

Kindly, keep us updated about the results you get, as this functionality has been added just recently, and we're more than willing to get the relevant feedback regarding it. Thanks.

[GarthH]

Hi

I am still messing around trying to get my GFS setup properly and have the issues mentioned by OP

Would an alternative to scripting be a second backup job (for each job) that is set to do an active full and only run on saturday
Then use this job as the source for the job to tape? With only 1 restore point set, it archives to tape every week and then overwrites itself for the next weekly
And your last weekly of the month you keep as your monthly?

[foggy]

Would an alternative to scripting be a second backup job (for each job) that is set to do an active full and only run on saturday

Yep, this is also a viable approach, however requiring the production VMs to be disturbed twice.

[techguy615]

Update: I have been a bit detained in reporting back the results of this. It appears the reverse incremental will give me what I need with a full backup writing to disk at the end of the day. The VM I am working with is approximately 2TB in size, so it takes a while to complete. Removal of the snapshot used is painfully slow, as it took 3hrs 42mins to be removed before the job showed as completed and the backup copy could begin. I just changed to the reverse incremental over the weekend. That seems to have worked fine writing to my NAS device. I am now waiting for the Backup Copy job to finish writing to my 4TB locally attached drive. The hope is that I will end up with the full backup plus one other restore point to take to a vault. If all goes well the job will hopefully be done by the end of the day.

[techguy615]

Ok, so it is 10.5 hours that my backup copy is running and it appears that I am only 36% complete. It is a great idea, but I am not so sure it will work for me if it takes 24+ hours to copy from a NAS device to a locally attached Hard drive. Showing my bottleneck being the source, I guess that means the NAS device I am copying it from is sucking wind. I am a bit perplexed though because it is a 12 spindle device and it doesn't seem the network card is being maxed out at all. I would think the local eSATA hard drive I am copying to would be the weakest link. Keep in mind this is from one server to another in the same rack with a gigabit connection. Any ideas? I just want to get a full backup on a drive to take it off-site.

[veremin]

Yep, it seems that your "source" storage can't provide data any faster.

However, be aware that there is always bottleneck in every deployment. So, just message regarding source being the bottleneck doesn't indicate anything, more interesting the full bottleneck statistics with percent distribution and the total backup copy job speed.

Thanks.

[MaartenK]

I too am waiting for this functionality...

example:

We keen 5 points of retention on all our backup jobs which run daily, monday to friday (and use archive jobs to keep archive retention).

We run MONTHLY backup to tape jobs for all our backups.

With 5 retention points set on the jobs, we normally have 2 full backups on disk at any one time (Forward Incremental - Dedupe unit). Because we only run the tape job monthly, each time it runs, both full backups are new to the tape job, and it writes both. Some of these backups are 2TB+ and this takes too long to write to tape.

sorry to hijack this topic but it was an interesting one.

We almost have the same configuration but with a longer retention to disk

Monday to Thursday Incremental and Friday a Full. With a retention on disk with 21 restore points.
A Copy job is made to another offsite repository to keep 14 restore points, 4 week, 12 months.
But we must keep data on tape (company policy). So we have even more Full backups on the primary site. This means all .vbk files will be put on tape (we run a backup to tape on Saturday). And we also have 2TB+ vbk files. so 6 .vbk files will be backupped on tape .

An option like "only backup latest Full backup to tape" would be a welcome one.

Otherwise you could use Incremental > synthetic full > with transformation. Then only one .vbk will be left on onsite and the backup to tape only sees one .vbk file? Correct me if i’m wrong

[veremin]

Otherwise you could use Incremental > synthetic full > with transformation. Then only one .vbk will be left on onsite and the backup to tape only sees one .vbk file?

With such settings, tape job will, indeed, see and copy only one .vbk file. Actually, you might as well switch to reversed incremental mode in order to have only one ("the latest") backup on disk which, then, will be copied to tape mediums.

Thanks.

[MaartenK]

With such settings, tape job will, indeed, see and copy only one .vbk file. Actually, you might as well switch to reversed incremental mode in order to have only one ("the latest") backup on disk which, then, will be copied to tape mediums.

Thanks.

thank you for your reply. That would be an option. But that doesn’t have any effect on how the "backup Copy job" works. This job still processes the latest backups to the offsite repository?

under advanced > incremental the text says "Recommanded for backup to tape"? This acutely means that reverse backups are better for backup to tape then a more traditional way with incremental and synthetic fulls?

The option "last full backup to tape" would be the best . Hope Veeam will consider this option in a feature release

[veremin]

This job still processes the latest backups to the offsite repository?

The mode used by "source" backup job doesn't affect backup copy job, as it creates restore points in "target" location, using its own synthetic logic.

under advanced > incremental the text says "Recommanded for backup to tape"?

It depends on the goals customer is trying to achieve. Generally, it's recommended to utilize forward incremental mode in case of tapes, since, this way, only incremental data, not the full backup, will be transferred on everyday basis.

However, your case appears to be different from the one mentioned above, and, thus, reversed incremental suites your needs better.

The option "last full backup to tape" would be the best . Hope Veeam will consider this option in a feature release

Yes, we do consider adding such or similar functionality in one of next product releases.

Thanks for the feedback; much appreciated.

[sbbots]

The option "last full backup to tape" would be the best . Hope Veeam will consider this option in a feature release

And hopefully "Only most recent vbk file" from a backup repository for "Files to Tape" jobs. I have a script that does this in a round-about way but built-in functionality wouldn't hurt. One can dream of a brighter, less-scripted future

[veremin]

Hi, Matt,

You mean sometimes it happens that there are more than two full backups in given directory that haven't been copied yet to tape mediums by file to tape job, and you want to copy only the latest file, right?

Thanks.