Discussions related to exporting backups to tape and backing up directly to tape.
Post Reply
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by foggy »

baatch wrote:Is it then possible to start a Veeam tape job with powershell?
I believe you can use Start-VBRJob cmdlet to do that (as in the script provided above).
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

baatch wrote:Ok thanks for the answear. Is it then possible to start a Veeam tape job with powershell?
Yep, the script should be pretty simple.

First, you get the corresponding tape job. Then, you start it.

Code: Select all

Get-VBRTapeJob -name "Name of your Tape Job" | Start-VBRJob 
Thanks.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

mhaynesVCI wrote:As much as I appreciate this script and the effort by the team it doesn't help us poor SOBs that are using a network repository to store our backups as hardlinks are not supported.
Then, you can automate slightly different approach. The one described above might do the trick for you.

Thanks
Stuzoo72
Novice
Posts: 4
Liked: never
Joined: Nov 07, 2013 1:55 am
Full Name: Stuart Montgomery
Location: Sydney
Contact:

[MERGED] : Enhancement Request: Single copy of backup to tap

Post by Stuzoo72 »

Currently when we use "Tape Job - Backups" and select a backup job it places all the restore points on to tape.

For jobs with a large retention policy this can be a lot of restore points and a lot of duplicated data. This results in many tapes.

I would like Veeam to add a couple of options for "tape jobs - Backups".
Option 1: Place most recent Full backup to tape only. This would place the most recent full backup to tape, whether this be a reverse incremental, active full or old active full with increments.

Option 2: Place most recent Active Full backup to tape.

These options would allow only the data we require for a single point in time backup to be placed on tape and so reduce the costs associated with having to use tape.

Thanks
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Hi, Stuart,

Yep, you're right. For now there is no functionality that allows you to copy only the latest full backup to tape. So, one of the options you have is to use the file to tape job, as well, hardlinks. The way this process can be automated is described in this post.

Thanks.
sbbots
Enthusiast
Posts: 96
Liked: 25 times
Joined: Aug 16, 2013 5:44 pm
Full Name: Matt B

Re: Backup to Tape only last backup with reversed incrementa

Post by sbbots » 1 person likes this post

v.Eremin wrote:Otherwise, you can create a folder called “Latest VBK” and specify it as a source for “file to tape” job. Within a script you should find the latest .vbk file, copy it to the aforesaid folder, execute corresponding tape job and delete .vbk file once the job is done. Thanks.
This is exactly what I do and it works perfectly. I use a simple BAT file in Windows Task Scheduler to accomplish this.

Create a new "Files-to-Tape" job with no schedule pointing to your "Latest VBK" folder and copy the job ID at the end of the setup. Next, copy the commands below to a text file, change the RED variables to your setup (including the job ID) and save as a BAT file. Lastly, create a task in Windows Task Scheduler to run the BAT file at the time/date of your choosing. If you want to delete the files after the tape job is complete just put a delete command at the end, but ***make sure you set a delay long enough to allow the tape job to finish before deleting the files***. In my setup I choose to manually delete the files myself.


::Delete all files in directory
del /F /Q "F:\Latest VBK\*.*"

::Copy latest VBK to directory
@echo off
echo. |date |find "current" >> "F:\Latest VBK\Log.txt"
echo. |time |find "current" >> "F:\Latest VBK\Log.txt"
set "source=F:\Backups_Daily\VMHOST01"
set "dest=F:\Latest VBK"
pushd "%source%" ||(echo.Source does not exist&pause&goto EOF)
for /f "tokens=*" %%f in ('dir /A-D /OD /B *.vbk') Do set "file=%%f"
popd
xcopy /d /i "%source%\%file%" "%dest%\" >> "F:\Latest VBK\Log.txt"

::Copy latest Veeam Config to directory
@echo off
set "source=F:\Backups_Daily\VeeamConfigBackup\VMHOST01"
set "dest=F:\Latest VBK"
pushd "%source%" ||(echo.Source does not exist&pause&goto EOF)
for /f "tokens=*" %%f in ('dir /A-D /OD /B *.bco') Do set "file=%%f"
popd
xcopy /d /i "%source%\%file%" "%dest%\" >> "F:\Latest VBK\Log.txt"
echo. |time |find "current" >> "F:\Latest VBK\Log.txt"

::Run Veeam backup job
"C:\Program Files\Veeam\Backup and Replication\Backup\Veeam.Backup.Manager.exe" backup c2d493aa-e310-4f6d-a15c-88d1163e67dc
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by mongie »

I would also like an option to only copy the latest full backup to tape, and not all vbk's in the backup chain since the older vbk is already on an older tape. And space and time are often rare on tapes.
I too am waiting for this functionality...

example:

We keen 5 points of retention on all our backup jobs which run daily, monday to friday (and use archive jobs to keep archive retention).

We run MONTHLY backup to tape jobs for all our backups.

With 5 retention points set on the jobs, we normally have 2 full backups on disk at any one time (Forward Incremental - Dedupe unit). Because we only run the tape job monthly, each time it runs, both full backups are new to the tape job, and it writes both. Some of these backups are 2TB+ and this takes too long to write to tape.
Dima P.
Product Manager
Posts: 14726
Liked: 1707 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by Dima P. »

Alex,
Thank you for providing the valuable use case! We added this request to tape feature list for the upcoming versions.
chadtandy
Enthusiast
Posts: 36
Liked: never
Joined: Feb 09, 2010 8:26 pm
Full Name: Chad
Contact:

[MERGED] Veeam v7r2 copy to tape scheduling question

Post by chadtandy »

Hi all, we're on Veeam v7r2 and are going to soon be moving from copying Veeam files to tape with Commvault to having Veeam copy directly to tape itself. I've poked around with the scheduling a bit but seem to be missing something. I just want to copy to tape once a week, and I only want to copy the latest full, not the entire backup chain. How do I schedule this?

Thanks, and sorry for my ignorance...

-Chad
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Hi, Chad. Your post has been merged into existing discussion.

Currently, there is no such functionality, so, you have to use workarounds (scripts) mentioned above in order to get what you're after. However, we consider adding this functionality in one of the next product releases.

Thanks.
smallsheep
Lurker
Posts: 2
Liked: never
Joined: Jan 07, 2014 11:14 am
Full Name: Matthew O'Brien
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by smallsheep »

I can add my name to the list of those waiting for this functionality. I have a customer who does reverse incremental with backup to tape and they were having instances of two full *.vbk files being backed up in a night which caused the job to spill over onto 2 tapes, they had a standalone drive rather than an auto-changer so this was an issue for them. If it was possible to limit Veeam to backup only the latest full it would be very useful indeed. :)
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

I would like to see this as well. I am toying around with a simple way to do this myself. I am trying to eliminate tapes entirely from the process. I currently have 4-4TB drives attached to a hot swap hard drive bay. I am in the process of migrating my file shares to a VM in order to finally give Symantec the boot. So as for now I am attempting to use good old robocopy to do this.

(Assuming E:\ is the local destination)

Batch File Contents:
------------------------
del E:\*.* /q
robocopy <VeeamRepository#1> E:\ *.vbk /MAXAGE:1 /LOG:C:\RobocopyLogs\Backup.log (Copy files from the Main repository to the local E:\ drive with the vbk extension, excluded ANY files that are older than one day).

------------------------
This way I get ONLY the newest .vbk file written to my local E:\ drive and can take it offsite. I have added the batch file to my "Post Job Activity" in the main backup job. I figure from there someone could copy it to tape if needed or possibly incorporate this method to get the files to tape somehow. This is all just theory though. I am about to run a backup and see what happens. I am hoping it will work without too much trouble. I will post back with the results.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Hi, Chad, haven't you tried to use backup copy job, instead of Robocopy? We have introduced recently some code improvements that allow backup copy job to work with rotated medias; might be worth taking a look.

Thanks.
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

I have been playing around with it a bit. I am thinking if I tell it to only keep one restore point it should get the latest vbk file only and not the incremental backup files. Robocopy isn't a preferred way to do any of this of course. I was just looking for a workaround to get something to disk for off site. Everything else seems to be working fine as far as I can tell.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

I am thinking if I tell it to only keep one restore point it should get the latest vbk file only and not the incremental backup files.
First of all, the backup copy job doesn't t copy backup file as a whole, but rather synthetically creates required restore points in "target" location from VM data in source backup repositories. Also, with the backup copy job the least number of restore points that can be set is 2. So, with the backup copy job you will always have at least 2 restore points (1 .vbk, 1 .vib). However, with the referenced regkey, backup copy job should work perfectly with rotated media.

So, it's certainly worth trying.

Thanks.
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

I have reconfigured my job to use the reverse incremental. I did see where the job was limited to only letting me select 2 as the number of restore points. I have also noticed if I entered a 1 manually instead of clicking the up/down arrows it doesn't complain about it though. That might not be a good thing though. As for the registry entry you are referencing, I am apparently missing it. Nothing like showing up late for the game.

Thanks for all of the help thus far. I am very new to the product at this point.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

techguy615 wrote: As for the registry entry you are referencing, I am apparently missing it. Nothing like showing up late for the game.
No problem. This is the registry key I was talking about. This regkey was designed specifically for rotated media scenario, and, once implemented, it will enable automatic cleanup of the backup repository from all existing files if any backup file from the latest full backup chain is missing. So, it might be worth taking a look.

Thanks.
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

Sounds good. I downloaded the patch and added the reg keys. Will see what happens.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Kindly, keep us updated about the results you get, as this functionality has been added just recently, and we're more than willing to get the relevant feedback regarding it. Thanks.
GarthH
Novice
Posts: 7
Liked: never
Joined: Jan 22, 2014 1:56 am
Full Name: Garth Horsburgh
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by GarthH »

Hi

I am still messing around trying to get my GFS setup properly and have the issues mentioned by OP

Would an alternative to scripting be a second backup job (for each job) that is set to do an active full and only run on saturday
Then use this job as the source for the job to tape? With only 1 restore point set, it archives to tape every week and then overwrites itself for the next weekly
And your last weekly of the month you keep as your monthly?
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by foggy »

GarthH wrote:Would an alternative to scripting be a second backup job (for each job) that is set to do an active full and only run on saturday
Yep, this is also a viable approach, however requiring the production VMs to be disturbed twice.
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

Update: I have been a bit detained in reporting back the results of this. It appears the reverse incremental will give me what I need with a full backup writing to disk at the end of the day. The VM I am working with is approximately 2TB in size, so it takes a while to complete. Removal of the snapshot used is painfully slow, as it took 3hrs 42mins to be removed before the job showed as completed and the backup copy could begin. I just changed to the reverse incremental over the weekend. That seems to have worked fine writing to my NAS device. I am now waiting for the Backup Copy job to finish writing to my 4TB locally attached drive. The hope is that I will end up with the full backup plus one other restore point to take to a vault. If all goes well the job will hopefully be done by the end of the day.
techguy615
Novice
Posts: 6
Liked: never
Joined: Jan 15, 2014 6:47 pm
Full Name: Chad Webster
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by techguy615 »

Ok, so it is 10.5 hours that my backup copy is running and it appears that I am only 36% complete. It is a great idea, but I am not so sure it will work for me if it takes 24+ hours to copy from a NAS device to a locally attached Hard drive. Showing my bottleneck being the source, I guess that means the NAS device I am copying it from is sucking wind. I am a bit perplexed though because it is a 12 spindle device and it doesn't seem the network card is being maxed out at all. I would think the local eSATA hard drive I am copying to would be the weakest link. Keep in mind this is from one server to another in the same rack with a gigabit connection. Any ideas? I just want to get a full backup on a drive to take it off-site.
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Yep, it seems that your "source" storage can't provide data any faster.

However, be aware that there is always bottleneck in every deployment. So, just message regarding source being the bottleneck doesn't indicate anything, more interesting the full bottleneck statistics with percent distribution and the total backup copy job speed.

Thanks.
MaartenK
Service Provider
Posts: 28
Liked: 6 times
Joined: Aug 25, 2011 7:45 am
Full Name: Maarten Kalkema
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by MaartenK »

mongie wrote: I too am waiting for this functionality...

example:

We keen 5 points of retention on all our backup jobs which run daily, monday to friday (and use archive jobs to keep archive retention).

We run MONTHLY backup to tape jobs for all our backups.

With 5 retention points set on the jobs, we normally have 2 full backups on disk at any one time (Forward Incremental - Dedupe unit). Because we only run the tape job monthly, each time it runs, both full backups are new to the tape job, and it writes both. Some of these backups are 2TB+ and this takes too long to write to tape.
sorry to hijack this topic but it was an interesting one.

We almost have the same configuration but with a longer retention to disk

Monday to Thursday Incremental and Friday a Full. With a retention on disk with 21 restore points.
A Copy job is made to another offsite repository to keep 14 restore points, 4 week, 12 months.
But we must keep data on tape (company policy). So we have even more Full backups on the primary site. This means all .vbk files will be put on tape (we run a backup to tape on Saturday). And we also have 2TB+ vbk files. so 6 .vbk files will be backupped on tape :shock:.

An option like "only backup latest Full backup to tape" would be a welcome one.

Otherwise you could use Incremental > synthetic full > with transformation. Then only one .vbk will be left on onsite and the backup to tape only sees one .vbk file? Correct me if i’m wrong
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Otherwise you could use Incremental > synthetic full > with transformation. Then only one .vbk will be left on onsite and the backup to tape only sees one .vbk file?
With such settings, tape job will, indeed, see and copy only one .vbk file. Actually, you might as well switch to reversed incremental mode in order to have only one ("the latest") backup on disk which, then, will be copied to tape mediums.

Thanks.
MaartenK
Service Provider
Posts: 28
Liked: 6 times
Joined: Aug 25, 2011 7:45 am
Full Name: Maarten Kalkema
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by MaartenK »

v.Eremin wrote: With such settings, tape job will, indeed, see and copy only one .vbk file. Actually, you might as well switch to reversed incremental mode in order to have only one ("the latest") backup on disk which, then, will be copied to tape mediums.

Thanks.
thank you for your reply. That would be an option. But that doesn’t have any effect on how the "backup Copy job" works. This job still processes the latest backups to the offsite repository?

under advanced > incremental the text says "Recommanded for backup to tape"? This acutely means that reverse backups are better for backup to tape then a more traditional way with incremental and synthetic fulls?

The option "last full backup to tape" would be the best :wink:. Hope Veeam will consider this option in a feature release
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

This job still processes the latest backups to the offsite repository?
The mode used by "source" backup job doesn't affect backup copy job, as it creates restore points in "target" location, using its own synthetic logic.
under advanced > incremental the text says "Recommanded for backup to tape"?
It depends on the goals customer is trying to achieve. Generally, it's recommended to utilize forward incremental mode in case of tapes, since, this way, only incremental data, not the full backup, will be transferred on everyday basis.

However, your case appears to be different from the one mentioned above, and, thus, reversed incremental suites your needs better.
The option "last full backup to tape" would be the best :wink:. Hope Veeam will consider this option in a feature release
Yes, we do consider adding such or similar functionality in one of next product releases.

Thanks for the feedback; much appreciated.
sbbots
Enthusiast
Posts: 96
Liked: 25 times
Joined: Aug 16, 2013 5:44 pm
Full Name: Matt B

Re: Backup to Tape only last backup with reversed incrementa

Post by sbbots »

MaartenK wrote:The option "last full backup to tape" would be the best :wink:. Hope Veeam will consider this option in a feature release
And hopefully "Only most recent vbk file" from a backup repository for "Files to Tape" jobs. I have a script that does this in a round-about way but built-in functionality wouldn't hurt. One can dream of a brighter, less-scripted future :wink:
veremin
Product Manager
Posts: 20415
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup to Tape only last backup with reversed incrementa

Post by veremin »

Hi, Matt,

You mean sometimes it happens that there are more than two full backups in given directory that haven't been copied yet to tape mediums by file to tape job, and you want to copy only the latest file, right?

Thanks.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 5 guests