Backup files on Exagrid repositories randomly not available

[zuldan]

Veeam B&R: v9.5 U1
Exagrid: EX13000E v4.8.0.216
Case: 02069622

Randomly throughout the week files on Exagrid repositories become unavailable. The Veeam support person thinks the issue is due to a bad tape. The case has since been closed. I'm not so sure this is a bad tape issue but who am I to argue against the expert so I'm looking for a workaround to the issue we're having.



Simple rescanning the repository makes the backup files available again (EG Get-VBRBackupRepository -Name 'Exagrid_03_01' | Sync-VBRBackupRepository).



The problem is when the tape job for the backup job starts, it's unable to locate the files.



So I wrote a little script that generates Powershell scripts to be run by tape jobs that will refresh Exagrid repositories before the job starts.

Code: Select all

$VeeamScriptFolder = 'C:\Scripts\VeeamRepositoryRefresh'
$BackupRepositoryExclude = '_Copy'

Add-PSSnapin -Name VeeamPSSnapIn

# Create Repository Refresh script files
$BackupRepositoryList = Get-VBRBackupRepository | Where-Object -FilterScript { $PSItem.Name -notlike "*$BackupRepositoryExclude*" }


if (-not(Test-Path -Path $VeeamScriptFolder))
{
    $null = New-Item -Path $VeeamScriptFolder -ItemType Directory
}

# Generate Powershell script files
foreach ($BackupRepository in $BackupRepositoryList.Name)
{
    $VeeamScript = @"
Add-PSSnapin -Name VeeamPSSnapIn
Get-VBRBackupRepository -Name '$BackupRepository' | Sync-VBRBackupRepository | Out-File -FilePath '$VeeamScriptFolder\$BackupRepository.log'
Start-Sleep -Seconds 20
"@
    $VeeamScript | Out-File -FilePath "$VeeamScriptFolder\$BackupRepository.ps1"
}

Write-Host -Object 'Retrieving Backup Job list'
$BackupJobList = Get-VBRJob | Where-Object -FilterScript { $PSItem.JobTargetType -eq 'Backup' }

Write-Host -Object 'Retrieving Tape Job list'
$TapeJobList = Get-VBRTapeJob

foreach ($TapeJob in $TapeJobList)
{
    Write-Host -Object "Processing - $($TapeJob.Name)"

    # Find linked Backup Job
    $BackupJob = $BackupJobList | Where-Object -FilterScript { $PSItem.Id -eq $TapeJob.ScheduleOptions.JobId }

    # Find repository of Backup Job
    $Repository = ($BackupRepositoryList | Where-Object -FilterScript { $PSItem.Id -eq $BackupJob.Info.TargetRepositoryId }).Name

    # Set script
    $ScriptOptions = New-VBRJobScriptOptions -PreScriptEnabled -PreCommand "$VeeamScriptFolder\$Repository.ps1"
    $TapeJob | Set-VBRBackupToTapeJob -JobScriptOptions $ScriptOptions
}

The script executes perfectly when the tape job starts and the Exagrid repository is rescanned. The problem is when you have 3 tape drives with 20 tapes jobs, the time that the tape job starts is different from when the tape job actually gets to use a tape drive. The time difference might be 20 hours and in that time an Exagrids backup files might become unavailable again. So my question is how do I execute a script just before the tape job REALLY starts?

[PTide]

Hi,

I'm not so sure this is a bad tape issue but<...>

Me, neither. I suppose that the tape corruption is just a coincidence and is not related to the original problem, so I advise you to escalate the case in order to let our engineers to find the reason of Exagrid repo becoming unavailable.

Thanks

[zuldan]

Just a quick update, support is still working on the case.

As a work around I created a Powershell script that runs as a scheduled task every 30 minutes to refresh all the Exagrid repositories. It's working perfectly and haven't had a single tape job fail since.

Code: Select all

Add-PSSnapin -Name VeeamPSSnapIn

$RepositoryList = Get-VBRBackupRepository | Where-Object -FilterScript { $PSItem.Name -notlike '*_Copy' -and $PSItem.Name -notlike 'Default Backup Repository' }

foreach ($Repository in $RepositoryList) {
    $Repository | Sync-VBRBackupRepository | Out-File -FilePath "C:\Scripts\VeeamRepositoryRefresh\$($Repository.Name).log"
    Sleep -Seconds 2
}