Guest File System Indexing

[dsimpkins]

I've done some searching on these forums and Google but can't find quite the answer im looking for, only related info.
I've also looked at the online manual, but again, can't quite get my hands on the answers I'm after

What exactly does the Guest File System Indexing do? The menu tells me it "creates catalog of guest files to enable browsing, searching and 1-click restore of individual files"... okay, that sounds handy. But then it continues "indexing is optional and not required to restore instant file level recoveries". Could someone confirm what exactly the advantage is here - is it purely a speed thing? is it a case of if i want to restore a file and i know the name but don't know the location then i can search (as opposed to having to know it's location for the 'non-indexed' file-level recovery)?

The second part to my question is regarding the folder size. Again, no one has quite answered this question - but apologies if i missed it: if i want to reduce the amount of space my VRBCatalog folder is consuming, can i simply delete the older index folders? here's an example: FileServer01 backs up once a day, and there's a 30 day retention period (or 30 increments) but I've got 90 of these index folders. I'm only interested in the indexes of the backups i have, can i simply delete the folders of the oldest indexes? or is there a 'proper' way of removing these, whilst retaining the functionality of the guest file system indexing?

thanks for your help, sorry if something obvious has been missed on my part.

Daniel

[Vitaliy S.]

is it a case of if i want to restore a file and i know the name but don't know the location then i can search (as opposed to having to know it's location for the 'non-indexed' file-level recovery)?

Yes, that's correct. Indexing is not required, if I know that file location and can find the required restore points yourself.

but apologies if i missed it: if i want to reduce the amount of space my VRBCatalog folder is consuming, can i simply delete the older index folders?

I guess you can do that, but you can also enable compression on the volume hosting your VBRCatalog data to use less disk space.

See these topics for further reading: cleaning up vbrcatalog folder and VBRCatalog folder growing in size

[dsimpkins]

Thanks for that, i read the second one of your links already, which didn't really answer my question: is it ok or SAFE to manually delete the older indexes, ie, are the newer indexes tied to the older ones, and if i delete them will something break? your answer of "i guess so" seems to indicate it's not safe?!

In the mean time, i've changed the retention period to 1 month (instead of 3) as per the first of your links, and added 10GB to that volume to be safe.

thanks

[veremin]

As far as I know, you can manually delete index files you don’t need any longer without any consequences to newer ones.

Nevertheless, if you’re still concerned with problem of older index files being tied to newer ones, you might want to just move your obsolete files to new location and see what happens when you try to use "Search" functionality in Enterprise Manager.

Hope this helps.
Thanks.

[Vitaliy S.]

Yes, it is safe to delete these files manually.

[sergeadam]

I have my retention period set to 30 days in Enterprise manager. My VBRCatalog folder is over 250GB. Under the index/machine folders, each machine has nearly 175 forlders.

How do I clean this up to retain only 30 days of catalog?

[dsimpkins]

I have my retention period set to 30 days in Enterprise manager. My VBRCatalog folder is over 250GB. Under the index/machine folders, each machine has nearly 175 forlders.

How do I clean this up to retain only 30 days of catalog?

This is kinda what i was trying to ask,

Yes, it is safe to delete these files manually.

seems this is the answer..?

[veremin]

seems this is the answer..?

Yep, you’re right.

But as I’ve said previously, if you’re a little bit anxious about potential consequences of such decision you might want to temporarily move unneeded catalog-files to another location before final deletion and test “Search” functionality to see whether it works smoothly or not.

Hope this helps.
Thanks.

[sergeadam]

The dates on the files and folders are all over the place. There does not seem to be an easy way to identify which are used and which are not.

[dsimpkins]

The dates on the files and folders are all over the place. There does not seem to be an easy way to identify which are used and which are not.

but the creation/modified dates within Windows Explorer would be accurate, yeah?

[sergeadam]

No they're not. Unless I'm missing a major piece of the puzzle. If I have guest file indexing off for my jobs, can I remove all the data in the VBRCatalog/index folder?

[foggy]

Yes, unless you need search & 1-click restore functionality in Enterprise Manager.

[sergeadam]

If I delete that folder, will it rebuild as needed? Just looking at getting a clean copy.

[foggy]

If you enable gust file system indexing on a job, index data will be generated during the following job runs and placed into that folder.

[jburkegb]

I have an issue with our veam server, the folder VBRCatalog\Journal is growing on a daily basis, it is currently at 60gb I have set the retention period on Enterprise manager for guest file catalog to 1 month this fixed the issue with the index's and event history was set to 3 months. guest file indexing has been disabled on all servers apart from 1 job that includes our file servers. But the journal folder just keeps growing. Is it safe to delete any of these files?
What is creating the files and how can this be limited to the amount of space/time to save the files?

We are running Veeam backup and Replication 6.5.0.128, Enterprise Manager 6.5.109

Any help or advice would be appreciated

[veremin]

Hi, Jason, you’ve been merged into existing discussion regarding VBRCatalog folder cleaning. So, kindly check the answers provided above and should any other questions arise, feel free to ask.

Thanks.

[jburkegb]

Hi v.Eremin

My question isnt related to the index folder this is working as it should, it is related to the journal folder , this is consming vast amounts of space 2.7gb per day I am only indexing 3 servers. What is the journal folder userd for? Is there any way to cap the size or the number of instances that it keeps

[veremin]

Journal folder contains contents presented to Search Server for indexing (text files).

If it’s Standard Edition that is being used, EM will preserve only index files of existing backups that currently are available on backup repositories.

In case of Enterprise one, EM will also keep the index files for backups that are no longer present in backup repositories, such as moved away to tape/offsite appliance, etc. In other words, in case of Entreprise edition, catalogue will be also kept for archived backups in accordance to the retention period specified in EM settings (EM -> Configuration -> Settings).

Thanks.

[jburkegb]

Thank Vladmir

This still wouldnt explain why all of a sudden it starts to take up this amount of disk space, previously we had indexing set on all backup jobs (around 50 vms 15 jobs) and we were never alerted regarding disk space. Now I have set retention to 1 month (minimum allowed for retention) and disabled indexing on all but 1 veam backup job containing 3 vms around 2tb in storage and its using upwards of 70gb including index folder. The system drive is 136gb drives with the os and veam taking up over 80%.

So in conclusion turn off indexing and delete these files?

Thanks

[foggy]

Or enable compression on that volume.

I would also recommend to open a case with our technical team, probably there was some other change performed that caused the folder size growth, logs will tell the reason.

[massimiliano.rizzi]

Hello experts,

we have a problem with the “C:\VBRCatalog” folder filling up the drive on a physical Veeam B&R server.

Since Veeam Backup Enterprise Manager is not being used, we disabled the guest file system indexing check box so that Veeam Backup & Replication will stop performing file indexing.

Is it safe to delete the entire “C:\VBRCatalog” folder ?

Thanks and Regards,

Massimiliano

[davidb1234]

I would like to remove all indexing data. None of our jobs use guest indexing and this c:\vbrcatalog folder takes up over 30GB. How can I disable this functionality completely or remove this? All of our backup jobs are set to not index guest file system.

[veremin]

Hi, David,

Your post has been merged into existing discussion regarding Catalog folder cleaning; so, kindly, see the answers provided above.

If you're not going to use indexing functionality or 1-click item restore (available in EM), you can safely delete all the data from this folder.

Thanks.

[ForestTom]

Hello!

We are having some issues with our Veeam Backup, mainly with the index backups at D:\VBRCatalog\Index\Machines

I have been looking around and thought the setting was in the Enterprise Manager console, Configuration > Settings > Guest File System Catalog, which is currently set as 1 month but there are backup points in this folder dating back to last year, have i missed something?

Sorry if this a simple fix! i believe it is okay to delete older folders without any side effects, but i would rather it be done automatically!

we are running V7.0.0.764

any help would be greatly appreciated!

[veremin]

Hi, Thomas,

Your post has been merged into existing discussion; please, take a look at the answers provided above. As mentioned, the obsolete, unused files can be deleted manually without any issues. Though, you can move them somewhere first and see whether anything is affected by this action. If not, proceed to complete deletion.

Thanks.

[siskos]

On our production system running Veeam Backup Enterprise and Veeam Backup & Replication we are running out of disk space.

It seems that the VBRCatalog/Journal folder keeps growing.
As far as I can see it creates a folder for every month of backups, at the moment we have 3 folders 2014_07 - 09 with each holding approx 25GB of data.

On these forums we have found that you can change the setting Guest File System Catalog but this did not help.
Currently the setting is at 1 month but the older folders are still on the filesystem, can we safely delete these older folders (2014_07 & 2014_08)?

Thanks in advance.

[veremin]

Hi,

Your post has been merged into existing discussion. Kindly, see the answers provided above regarding similar matter. More specifically, this post. Feel free to ask for additional clarification, if needed.

Thanks.

[btmaus]

I have some questions regarding Guest File Indexing:

1. When a Windows Server is backed up by Veeam with the Index option enabled, where does that indexing take place?
Is it within the VM being backed up?
If so, and I only specify the D drive to be indexed and nothing else, will the indexing files be stored on the D drive of that VM?

2. Where are the indexes kept, on the VBR server I assume?

3. I was testing indexing on some test servers, how do I remove those indexes now so that they don't show up in Enterprise Manager anymore?

[foggy]

1. When a Windows Server is backed up by Veeam with the Index option enabled, where does that indexing take place?
Is it within the VM being backed up?

Right.

If so, and I only specify the D drive to be indexed and nothing else, will the indexing files be stored on the D drive of that VM?

Indexes are stored inside VM only temporarily, until being sent to the Veeam Backup Catalog folder.

2. Where are the indexes kept, on the VBR server I assume?

Both on Veeam backup server and Veeam Backup Enterprise Manager (if installed on different server).

3. I was testing indexing on some test servers, how do I remove those indexes now so that they don't show up in Enterprise Manager anymore?

Just clean-up the Veeam Backup Catalog folder.

[btmaus]

Just clean-up the Veeam Backup Catalog folder.

Thanks foggy. So I can just delete those VM directories from that catalog folder?