v7 Backup Copy Job removable media (rotated drives)

[Vitaliy S.]

Arun, let me run your case by our dev and QC teams tomorrow to verify why the key is not working.

[zak2011]

Thanks Vitaliy. Appreciate your help!
Previously it was created under \HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication.
That time also the job used to fail and would succeed upon the next retry. So I was told to add it to Wow6432Node as the Veeam server is a 64bit OS.
Now I get the following error message every day the RDX is rotated even after the key is entered under
Wow6432Node
Perhaps I am missing something here.
Thanks

[zak2011]

The case has been escalated. I was told to create the registry keys at both locations now at \HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication and under the Wow6432 Node.
Then restart the Veeam server, create a new job and run the backup to RDX.
When the job runs first it works. However if i try to delete the vbk and rerun the job it fails with the following error
Error: Failed to find last point in backup 63cb57c1-cb92-4db0-aae4-10e5c7a4f0ad.
If i retry again, it starts working.
No clue whats happening...

[zak2011]

Does the registry key work in the case of a normal backup job writing to an RDX drive or does this work only for backup copy jobs writing to rotating media?

Thanks

[Gostev]

It is designed to work in both of these cases.

[tsightler]

It should work in either case as long as there is not a single incremental run in the chain. Most of the cases I've seen where it didn't work were caused by the fact that there was an incremental backup in the chain, for example due to the media not being swapped when expected. I'll admit I haven't tested this with anything other than Backup Copy Jobs in V7 though.

[zak2011]

Thanks Gostev and Tom for confirming this.
After the first successful backup job to the RDX, I tried deleting the vbk from the RDX and rerunning the job, it fails. However at the second retry it goes through. But for the Backup Copy jobs it seems to be working. It seems to be working once the key has been created at both \HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication and under the Wow6432 Node.
Would it be better also to use Tom's script also along with this?
It would be nice if it would work for the backup jobs also. I have contacted support about this and uploaded the logs today.

Thanks

[tsightler]

BTW, do you have your backup job configured for forward or reverse incremental? Seems like I remember having to configure the job to reverse incremental to make the key work some time ago.

The script will certainly work in any case and in my opinion makes the process more reliable as otherwise any cycle that somehow causes a VIB to be created will cause the rotation to fail. I could certainly enhance the script to provide some protection against deleting media too soon (for example if media is swapped from a previous day for restore) however.

[zak2011]

The job is configured for reverse incremental.
Right now since the backup job to the RDX works only for the second retry, I am using just a backup copy job till it is sorted out.
I am faced with a stranger problem now. Last night after the successful backup copy job run at 1:00 am, the user at the remote office gets almost 30 mails from Veeam every hour with the following error
________________________________________

Subject: [11/7/2013 8:21:32 AM][Failed] Offsite RDX-Backup Copy (1 VMs) 1 failed
BackupSync job: Offsite RDX-Backup Copy
Created by VEEAMSRV\Administrator at 11/6/2013 2:27:12 AM. Error
1 of 1 VMs processed
Wednesday, November 06, 2013 11:20:14 PM
Success 0 Start time 11:20:14 PM Total size 0.0 KB Backup size 16.1 MB Some VMs were not processed during the copy interval
Warning 0 End time 8:21:32 AM (+1) Data read 0.0 KB Dedupe 1.0x
Error 1 Duration 9:01:17 Transferred 0.0 KB Compression 1.0x
Details
Name Status Start time End time Size Read Transferred Duration Details
HELSESRV02 Error 8:21:21 AM 8:21:22 AM 0.0 KB 0.0 KB 0.0 KB 0:00:00 VM was not processed during the copy interval

[foggy]

I am faced with a stranger problem now. Last night after the successful backup copy job run at 1:00 am, the user at the remote office gets almost 30 mails from Veeam every hour with the following error

Arun, please also ask support for assistance in investigating this. And please kindly avoid posting about different issues in a single thread as doing so makes it confusing for the future readers. Thanks.

[zak2011]

To be honest..its really frustrating . I have been trying to contact Veeam support from morning by email, through portal.
Tried calling Veeam support number, entered the case no, no one picks up. Treid calling again and wait and wait and wait...absolutely no pick up.


I ran the following script which the first support guy gave me..

Code: Select all

$backupfilepath   = ($backup.GetStorages() | ?{$_.GetStorageType() -eq "vbk"}).FilePath

# Get Directory for deleting
$backupdirpath    = ($backup.GetStorages() | ?{$_.GetStorageType() -eq "vbk"}).DirPath

# If current active VBK is found on media then media has not been rotated
if (-not(Test-Path "$backupfilepath")) {
    # Current active VBK was not found indicating media has been changed
    # First DisableScheduler, If not then the Copy Job will Create a new Directory like $Jobname_1
    
    # Get-Job Object
    $CopyJob      = Get-VBRJob -Name $jobName
    
    # Disbale Scheduler
    $CopyJob.DisableScheduler()
    
    # then Wait until Job is Stopped (and is not running anymore)
    while (-not($CopyJob.IsStopped())) { 
        # Do Nothing 
    }
    
    # If stopped then Delete from VeeamGUI
    $backup.Delete()
    
    # Remove Backup Files from Media
    if (Test-Path "$backupdirpath") {
        # Remove whole Directory
        Remove-Item -Path "$backupdirpath" -Recurse
    }
    # Enable Scheduler
    $CopyJob.EnableScheduler()
}

After running this script..i got the following message
11/7/2013 2:31:54 AM :: Processing 'VM' Error: Client error: File does not exist. File: [F:\Offsite RDX-Backup Copy\Offsite RDX-Backup Copy2013-11-06T135443.vbk].
Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://Tar2Text]. CHMOD mask: [0].


What am I supposed to do then?

[foggy]

You tell me not to post on the forum!!!!

I'm not telling you not to post, but rather create separate topics for different issues instead. This will allow to keep threads not overfilled with irrelevant discussions and, hence, easier to track. Feel free to create a new topic for the backup copy job issue and do not forget to provide case ID. Thanks for understanding.

[zak2011]

Thanks foggy!. The case ID has been provided in this thread. Also the issue I am faced with is getting the backup copy job to work on removable media ( rotated drives).
Could anyone please advise if the script posted above is okay as I dont know when I get to hear from support.

Thanks

[foggy]

It's ok to discuss the rotated scenario in the current thread. I was talking about that one:

I am faced with a stranger problem now. Last night after the successful backup copy job run at 1:00 am, the user at the remote office gets almost 30 mails from Veeam every hour with the following error

[zak2011]

Sure....sorry about that post!
To get to the point. I ran the script against the backup copy job. It is not removing the previous vbk.
Anything possibly wrong with the script that I have posted here.

Thanks!

[zak2011]

Please ignore the one above.. I mean This script...

Code: Select all

# Import Veeam Snapin and we do not want Errors
asnp -Name VeeamPSSnapin -ErrorAction SilentlyContinue

# JobName here
$jobName          = "Offsite RDX-Backup Copy"

# Get Backup Specs
$backup           = Get-VBRBackup -Name $jobName

# Get FilePath
$backupfilepath   = ($backup.GetStorages() | ?{$_.GetStorageType() -eq "vbk"}).FilePath

# Get Directory for deleting
$backupdirpath    = ($backup.GetStorages() | ?{$_.GetStorageType() -eq "vbk"}).DirPath

# If current active VBK is found on media then media has not been rotated
if (-not(Test-Path "$backupfilepath")) {
    # Current active VBK was not found indicating media has been changed
    # First DisableScheduler, If not then the Copy Job will Create a new Directory like $Jobname_1
    
    # Get-Job Object
    $CopyJob      = Get-VBRJob -Name $jobName
    
    # Disable Scheduler
    $CopyJob.DisableScheduler()
    
    # then Wait until Job is Stopped (and is not running anymore)
    while (-not($CopyJob.IsStopped())) { 
        # Do Nothing 
    }
    
    # If stopped then Delete from VeeamGUI
    $backup.Delete()
    
    # Remove Backup Files from Media
    if (Test-Path "$backupdirpath") {
        # Remove whole Directory
        Remove-Item -Path "$backupdirpath" -Recurse
    }
    # Enable Scheduler
    $CopyJob.EnableScheduler()
}

[zak2011]

Update from Tier2 support is that the ForceCreateVBK key is currently being reviewed by R&D and therefore not to be used until a permanent soultion is found.
The script does not work for me as the remote office uses Veeam Standard edition so Veeam poweshell snap in is not installed.
The workaround suggested by support is the following

1. Create a backup job for each day of backup cycle.
2. Make sure the number of restore points to keep is set to 1 for every job: job properties > Storage
3. Schedule active full backup for every time certain job runs. You can do that under job properties > Storage > Advanced.
4. Schedule each job to run on a specific day of the week.


This way the vbk gets written to each RDX drive.

[foggy]

The script does not work for me as the remote office uses Veeam Standard edition so Veeam poweshell snap in is not installed.

Standard Edition includes a PowerShell Snap-in, you just have to select it manually in the setup wizard to get it installed.

[zak2011]

Thanks foggy. I somehow missed that. However Tier2 support advised not to use the scripts till a permanent fix was found for rotating media.
Thanks!

[foggy]

Sure, I've just mentioned that for the case you missed it.

[Kellen.Flood]

How does Veeam integrate with a Quantum RDX 8000? It is an 8 slot removable disk device with iSCSI target. Want to rotate the cartridges weekly

Do we need to take any special considerations into account for setup or jobs?

http://www.quantum.com/products/removab ... index.aspx

[veremin]

Hi, Kellen,

Currently, ForceCreateMissingVBK regkey that should be used in case of rotated drives doesn't seems to work properly. For more explanation and potential workarounds, kindly, see the answers provided above.

Thanks.

[Gostev]

The hot fix is now available through our support, however please keep in mind that it requires that R2 update is installed.

[spuntiga]

Hello Gostev,

and with this hot fix i do not need the powershell script, mentioned above?

[iamxCPx]

This is my scenario.
Every week I copy the full backup job to USB drive using Backup Copy Job method and this is what I do every week.

1. Disabled the "Backup Copy Job" job
2. Open the Veeam PowerShell
3. Run the following command:
$delete = Get-VBRBackup -Name "Job Name"
$delete.Delete()
4. Re-enabled the job.

According to Tier-2 Support, you should be able to create a script (PS1 File) and schedule it with Windows Scheduler for automation but I haven't done it yet.
Hoping someone here could share their experience dealing with scheduling a PS script. I haven't had time to google it.


This is the tips that I get from the Tier-2 Veeam Support.

PS1 File run first:

Get-VBRJob -Name "Job Name" | Disable-VBRJob

PS1 File run second (delayed 5-10 minutes later):

$delete = Get-VBRBackup -Name "Job Name"
$delete.Delete()
Get-VBRJob -Name "Job Name" | Enable-VBRJob

Hope that helps and hopefully Veeam will create a patch that it will be just a check box on the job.


PS.
The ForceCreateMissingVBK regkey doesn't work for me but the above steps does.
So if you contact Veeam and level 1 support suggests you to do the regkey, ask them to escalate it to Tier-2.

[foggy]

According to QC, the mentioned fix just restores the normal operation of the registry key. While the script is still required to workaround the existing backup copy increments that prevent normal implementation of this scenario.

[veremin]

Actually, there is no need to run two different PS scripts, as everything can be done within just one. All you need to do is to add Start-Sleep command that will wait for the specified interval before proceeding to the other lines. In the example provide below the interval is equal to 10 minutes.

Code: Select all

Get-VBRJob -Name "Job Name" | Disable-VBRJob
Start-sleep -s 600
$delete = Get-VBRBackup -Name "Job Name"
$delete.Delete()
Get-VBRJob -Name "Job Name" | Enable-VBRJob

Also, I've previously described the way PS scripts can be scheduled via Windows Scheduler; might be helpful.

Thanks.

[VolksMagic]

The hot fix is now available through our support, however please keep in mind that it requires that R2 update is installed.

hi there, i have the R2 update and the reg key installed and im still getting the "File not found. Agent failed to process method {Stg.OpenReadWrite}" error, can you tell me how i can get the patch please??

[Vitaliy S.]

Through our support team, just ask your engineer to provide it.

[farid.shahab]

As a workaround, I have had to use the mlink command to mount the removable drive as a local drive in Veeam server and then added it as a repository.
The new repository with a folder on the c drive pointing to the RDX was added as a Windows repository with the drive letter and path. I ran the job and it seems to be working this way.
Is this okay to do it from Veeam perspective?

Hi Zak,

can you please share the more details instruction for adding removable disk to backup repository, I have the same problem and I see that you have sorted out. thanks. Farid