Issues with Indexing Backups

[xeonjackson]

Hi all,

I opened a ticket with Veeam support, but I thought perhaps you all may also be able to offer assistance.

Here's the situation:

I am receiving this error: Failed to Index guest file system. VSSControl: Index Failed on two jobs backing up the same VM. I have five jobs that backup the different virtual drives/vmdk's of this VM. Three complete successfully with the indexing while two fail. In the each of the five backup jobs, only the child vmdks(drives) are being backed up, not the "C" "Windows/Administrative" drive. I could understand that the job was failing if the Windows vmdk wasn't present, but three of the five jobs backup and index their virtual drive vmdk without issue. Please let me know what is going on so I can rectify it.

Case # 02135002

Please help if you can.

[foggy]

System disk exclusion shouldn't affect guest file system indexing. Please continue working with your support engineer.

[adruet]

@xeonjackson are you running veeam backup and replication 9.5 update 2 ?

I just upgraded from 9.5 update 1 to update 2 and I have seen 23 warnings exactly like yours last night.

I am digging in the logs right now and will open a support case if necessary.

[foggy]

Hi Alex, I've checked the OP's case and it was closed after no response from the customer, so it does make sense to open your own case for investigation. Thanks.

[adruet]

Hi Alexander, after some investigation I did open a case: 02161438.
Thanks for your feedback.

[andriktr]

Looks that we also have same issue after update 2 install. Many VM's have same warning: "Failed to index guest file system. VSSControl: Index failed"
Any news regarding this issue?

[davecla]

Same here after update 2 applied
Just one server out of many though.

[DaveWatkins]

Just another me too here.

Installed Update 2 yesterday and overnight got 8 or so failed index warnings on jobs this morning. That's still a small fraction of our total VM's but markedly more warnings than I have ever seen otherwise.

Oddly, in our case it seems to mostly affect RDP servers

[adruet]

Veeam Support told me this is a new behavior because they now have a more robust logging engine which allows them to log what they didn't see before.

From a user side, the backups with the warning are ok, and we are able to restore.

I am still working with support to give them logs to work on the issue.

We do not have Remote Desktop hosts with the issue, but many different machines.

[Nicodonald]

Same here after update 2

[foggy]

For everybody who is seeing this behavior it is recommended to contact support, unless done already.

[davecla]

I've logged a support job too #02167192 - still just the one server.
Even if it's just informational, unless all my job reports are green then my day starts badly.

[JaySt]

same here. However, not consistently on the same VM each job run.

[davecla]

Anyone had a decent answer from support yet?
I've got another server now with the same error message and the one that was failing before now completes ok. No changes to the environment.

Support have indicated it's a permission problem with the account being used to run the indexer. I'm very skeptical......

[Nicodonald]

Same here, a server didn't had the warning since update and now it fail to index, the reverse is true too, a server had the warning since update and now it's ok.

[foggy]

I've checked a couple of cases mentioned above and they are still under investigation. Please open your own cases to confirm the issue and get notified about available solutions.

[foggy]

Latest state from R&D: the issue is confirmed, hotfix for Veeam B&R v9.5 u2 (104602) is available through support and is planned to be included into one of the next updates. Thanks.

[bucoops]

Hi All,

Another user with this issue. Just installed the hotfix and will see how that runs tonight. Our case ref is 02166440

[davecla]

Hotfix applied here. One time run through the backup cycle completed without warnings or errors.

[DaveWatkins]

Same here, fix looks good so far

[bucoops]

Agreed - my cycle just finished and no warnings/errors

Now to update the firmware on core switch 1 and then back to bed lol

[adruet]

Applied the patch too, and no more warnings !

[KeiichiKun]

Hi all,
a backup job of a sql server vm ends with this warning: "Failed to index guest file system. VSSControl: Index failed"
Log has this entry:

Code: Select all

VSSControl: Index failed   at Veeam.Backup.VssProvider.CVssControl.Index(Guid jobId, String[] includeFolders, String[] excludeFolders, String outputDir)
at Veeam.Backup.Core.CViGuestDiskIndexer.Veeam.Backup.Core.IGuestDiskIndexer.Index()
at Veeam.Backup.Core.CViGuestCatIndexer.TryToIndex(Guid jobId, CCatIndex& catIndex, Boolean& isGuestGroupMembersAvailable)
vss::CVeeamIndexingProvider::NetTryCompleteIndex. Failed to complete guest file system indexing in network mode.

All our backup proxies are in automatic mode.
Why could network mode give this error? For a network problem? Should we create another proxy and force it to use another mode instead of automatic and check again?
Thanks!

[DGrinev]

Hi,

Please contact the support team and let them review the logs to find out the real cause of the issue.
Also, when you are posting a technical issue you should share a case id number or your message will be deleted in 24 hours by moderators according to the rules of the forum. Thanks!

[KeiichiKun]

Sorry, #02303497

[albertwt]

a backup job of a sql server vm ends with this warning: "Failed to index guest file system. VSSControl: Index failed"
Log has this entry:

Code: Select all

VSSControl: Index failed   at Veeam.Backup.VssProvider.CVssControl.Index(Guid jobId, String[] includeFolders, String[] excludeFolders, String outputDir)
at Veeam.Backup.Core.CViGuestDiskIndexer.Veeam.Backup.Core.IGuestDiskIndexer.Index()
at Veeam.Backup.Core.CViGuestCatIndexer.TryToIndex(Guid jobId, CCatIndex& catIndex, Boolean& isGuestGroupMembersAvailable)
vss::CVeeamIndexingProvider::NetTryCompleteIndex. Failed to complete guest file system indexing in network mode.

All our backup proxies are in automatic mode.

So how did you go with this issue ?

[KeiichiKun]

Hi,
apologies for late reply.
I've solved disabling indexing because file level restore is available even if file indexing is disabled.
I've tried to move shadow copies from C: to a dedicated volume Z: but I got another error "The shadow copies of volume C: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied."
So I've choose drastic solution.

[rdkennedy]

Hello
Does hotfix 104602 only need to be applied to the Veeam server or does it need to be applied to the proxies as well?

[Vitaliy S.]

Hotifx should be installed on the Veeam backup server. Thanks!

[mtaggart]

We have this issue as well and we are on Update 3. We've opened a support case 02596079, but there's not been any resolution to it yet.

Help, anyone!?