Publish-VBRBackupContent can cause lots of errors to be logged and can leave orphan session

[AlexHeylin]

Arising from cloud-connect-backup-f43/vbr-vaw-val-to ... 80878.html I'm writing a script to mount the most recent RP for each backup and if it's an NTFS / REFS filesystem run CHKDSK /F against it to check for file system issues / corruption.

This means we're sequentially calling Publish-VBRBackupContent and Unpublish-VBRBackupContent on a number of RPs. During initial testing on two VBR servers we've observed a number of things which do not look right..

1. LOTS of DISK and NTFS errors logged in Windows while the RP is mounted. All are complaining about "hardware" issues.

Code: Select all

The IO operation at logical block address 0xbf3ce for Disk 4 (PDO name: \Device\00000143) failed due to a hardware error.

Code: Select all

The system failed to flush data to the transaction log. Corruption may occur in VolumeId: \\?\Volume{d4ae42f6-52fe-42b8-b1b0-73a6c15eab89}, DeviceName: \Device\HarddiskVolume156.
(The request failed due to a fatal device hardware error.)

1,000 - 134,000 instances of these messages in 12 hours.... not good and very concerning.

2. RPs mounted via Publish-VBRBackupContent may not show as mounted in the GUI, and may not show as mounted in the CLI either.
RPs can become stuck mounted / half-mounted and according to GUI and / or CLI it's not mounted and no publish session is running.

3. Unpublish-VBRBackupContent won't accept wildcard for session, and won't accept an array of sessions from pipeline.

Code: Select all

Get-VBRPublishedBackupContentSession | Unpublish-VBRBackupContent

Won't work, but the less intuitive

Code: Select all

Get-VBRPublishedBackupContentSession | foreach {Unpublish-VBRBackupContent -Session $_ }

Does seem to work. Is this deliberate, or oversight?

Does anyone have any experience or info to add on these?

One other question arises - is it possible to see if a backup / RP is Windows / Linux / Other - or what the filesystem(s) in it are without mounting (publishing) the RP? Mounted an RP containing a non-Windows file system to Windows fails anyway, but it'd be good to avoid using failures a as routing expected occurrence and make it an unexpected bad thing so the script can treat them as real errors.

Thanks

Case #05842336

[HannesK]

Hello Alex,
the case is from 27th January and the support case also seems to be V11... are you doing this on V11 or V12 RTM? And are you doing this on a normal backup server or on cloud connect with tenant backups?

Best regards,
Hannes

[AlexHeylin]

Hi HannesK,

Yes - I'm back to give you a headache again...

This is on v11 on our in-house production "all on one backup server" deployment (nothing like developing / testing on production right!? ). For now we're going with this approach to get to MVP, then we'll look at handling split-server deployments and automation.

We'll look at the cloud end later when all those upgrades are done and it becomes possible. We'll likely port this code across to do the cloud end. For now we're trying to find the problems with the "simple" approach of doing it locally before we start dealing with the additional challenges of the cloud.

Obviously we're not asking for support in writing the script (though any tips welcome) - we're looking for support on the unexpected behaviour arising from the PowerShell interface (which we understand is supported). Support have tried to bounce this off as "We don't support your scripts", hopefully they'll accept the distinction we've made in what we're asking for.

We've ended up having to reboot the VBR server to clear a stuck mount arising from Publish-VBRBackupContent. That's not something that's going to be acceptable in many cases this will be used, because the VBR server has other roles too (often the Hyper-V that's being backed up).

Thanks

Alex

[oleg.feoktistov]

Hi Alex,

Reviewed your case #05842336. Yes, I see that the errors you shared are actually technical and are not related to the way you approach your script design or mistakes that usually happen along the way (e.g. nulled variables, array passed instead of an object etc.). So if misunderstanding will continue to persist, don't hesitate to leverage "Talk to a Manager" option. From what you are describing, VBR logs will have to be checked for sure. Meanwhile, I'll be monitoring your ticket closely.

Now, about Unpublish-VBRBackupContent cmdlet design. I'm not in charge of powershell this long to tell whether this is deliberate, but I brought it up internally with QA to see if there were any initial concerns here with an array of sessions or pipeline workflow. If there weren't any, I don't see why not we add these 2 options in one of future releases.

Best regards,
Oleg

[AlexHeylin]

Hi Oleg,

I made plenty of mistakes in the script - just not that part of the script

Thank Oleg

[oleg.feoktistov]

Hi Alex,

Looks like making pipeline and arrays as input unavailable for Unpublish-VBRBackupContent was rather an oversight from our side.
I added it as a feature request.

Best regards,
Oleg

[AlexHeylin]

Thanks Oleg