VBRCatalog folder growing in size

[JohnG]

I installed Veeam a few days ago and it seems that my VBRCatalog is growing indiscriminately. I have added 15 gigs of space over the past 3 days to this disk (Veeam is running in a VM) and it is already full today. Since the VM runs out of space, jobs fail.

1) What does this folder do exactly?

2) Can I limit the size of it? (ie, can I periodically delete its contents?) What are some potential reasons for it growing so much? I have 4 jobs that run once an hour...

3) How can I move it to another folder/disk so I don't have to constantly grow the system disk? Short of reinstalling Veeam.

Thanks.

Edit: The 'index\Machines' subfolder is the one that seems to hold most of the data.

[Gostev]

The folder contains guest file system index, you have probably missed the setup wizard step explaining this (there was an option to set folder location different than default). Each restore point (and you are backing up every hour) produces its own index file. You may want to enable indexing for only certain guests, for which you really need it. You can refer to sticky FAQ topic on top of this forum for more info on guest file system indexing.

Yes, you should be able to move the folder by updating the corresponding registry key. I have not personally tested this though.

Thanks!

[JohnG]

I've decided to disable guest indexing completely on my backups (and deleted whatever indexes existed). If I ever do a restore, it will be a full VM and not individual guest files. Plus I have DPM backing up my actual file server data so I would use that to restore in such a situation.

Thanks.

[Gostev]

Actually, indexing is not required to restore individual files (see FAQ). You can restore individual files whether or not you perform guest file system indexing.

[bjamieson]

So... has an answer been found? I looked thru the registry and there are 5 or 6 references to the VBRCatalog. If I change the 2 keys pointing to the path, what about the other references? There must be an easier way to move the catalog?
Cheers

[Vitaliy S.]

Bruce, could you please contact our technical team for assistance with this? I guess changing registry keys is not enough to move a catalog folder.

[greendx]

Is there any way to predict / estimate disk space growth for the VBRCatalog folder?

In my example 7 day backup for 250 VMs. Currently have 40GB drive allocated for the folder and filling it up quickly. I want to know if it makes sense to add more disk space or to limit which VMs get indexed or maybe stop indexing them completely.

[Gostev]

Are you asking about Backup server, or about Enterprise Manager server?

[greendx]

Are you asking about Backup server, or about Enterprise Manager server?

I'm asking about Enterprise Manager server.

[Gostev]

For Enterprise Manager, you multiple that by your catalog retention policy (which can be adjusted in the Enterprise Manager configuration). For example, if it is 3 months, then 40GBx4x3 = about 500GB. Thanks.

[greendx]

Ok thanks. So I'm looking at the Catalog Settings in enterprise manager. Retention period is set to 60 months. I'd love to change it but the option is greyed out. Can't seem to find any information on this in the user guide.

[Gostev]

That means you are using Standard Edition, which is limited to keeping catalog on for backup files residing on disk. So, if you only keep 7 days worth of restore points on disk, the catalog will only keep data for these 7 days and should not grow further. Thanks.

[greendx]

Ok so I confused the issue. I do see that the Index subfolder under VBRCatalog does have 7 days worth and that's fine. The folder where I'm seeing growth is the Journal subfolder. I installed enterprise manager in May and I am seeing 3 subfolders under Journal: 2011_05, 2011_07 and 2011_08. These are the large folders. For example 2011_07 is 15GB and 2011_08, newest one which was just created a few days ago is already 5GB. Is there more details about these folders and how to go about managing their size?

[Gostev]

Journal folder contains contents presented to Search Server for indexing (text files). Amount of folders should be according to your catalog retention policy, as I explained above. Thanks!

[greendx]

Ok so then it should be ok to delete the folder from May? Shouldn't it have been cleaned up automatically since I am only doing 7 days retention?

[Gostev]

Job retention (7 days) has nothing to deal with this folder. Reading through my responses above, the data in this folder will be kept for the amount of months as specified in the Enterprise Manager, under Catalog settings. You may want to reduce that amount to make sure older data is deleted automatically. Thanks.

[greendx]

Yes and I replied that it is set to 60 months but greyed out on my end. To which you replied that is because I am running standard version of EM. So how do I go about reducing the 60 month setting to 1 month? Thanks.

[Gostev]

So, which Veeam Backup edition do you have in reality? Is it Standard? Then it should not keep this history at all. I may need to talk to the devs about this.

[greendx]

I have the standard version. Let me know if I should open a formal support ticket for this.

Thanks
Eugene

[gccit]

Found this post today while searching for info about my growing VBRcatalog\Index folder. We're working towards using Veeam as our ONLY backup product - eliminating Commvault and DPM2010. To do so, it's requiring much longer Veeam retention periods. 360 days or more in some instances. I love having indexing available when I'm looking for a specific file, but the disk space cost for the index long-term is too much really. I'd love if there was a way to delete indexes after a given # of days.

On a related note, if I disable indexing for a job that has previously had indexing enabled, will it purge them during the next backup run?

[Gostev]

v6 should reduce the size of the index in about 5 times.
No, job setting has no effect on already created catalog data.

[itdirector]

VBR 6.1
vSphere 5.0
Daily backup jobs - 30 day retension
Monthly backup jobs - 36 month retension

We have been running VBR for about 9 months now in 5 offices (5 VBR installations). In one of our offices (main office), the index folder (only 25 VMs) has grown to 95GB!
"D:\Program Files\Veeam\Backup and Replication\VBRCatalog\Index\Machines\"

- What is the correct procedure to move these existing indexes (the entire folder & GuestIndexData.zip files) to a larger drive (file copy + registry change?)
- What happens if we disable indexing for the jobs? Does that mean enterprise manager won't be able to search for individual files in backed up VMs anymore?

Thx

[Vitaliy S.]

1. Please see this KB article for the detailed procedure on how to accomplish this > http://cp.veeam.com/knowledgebase/artic ... ormat/html
2. EM will be able to search through the existing restore points that are already saved in the Catalog folder. All restore points that are done with VM indexing disabled, will not be displayed in the search results.

[itdirector]

Thx Vitaliy. That article states it is for VBR 5.x ; is it still valid for 6.1 & 6.5?

[Vitaliy S.]

Yes, it should!

[Andy]

Hi,

Please correct me if I'm wrong,
We running VEEAM B&R 7.0, I found there is 2 places stocking .idf files

1-C:\VBRCatalog
=> I believe this is for "Enable Guest file system indexing" in a backup job
which you can move by moving the folder to another storage and change the registry key HLM\SOFTWARE\Veeam\Veeam Backup Catalog\CatalogPath to reflect the physical location.

2-C:\Users\svc_operator\AppData\Local\Temp
=> this one is I believe for the "Gest file system catalog" setting on VEEAM Backup Enterprise Manager

In my observation, both of them taking a lot of space, sure depending on how long do you set the parameters.

I have a question, how can we move the 2rd catalog ? the one store/log "Gest file system catalog" ?

Many thanks.

[foggy]

Andy, please take a look at this post, should clearly answer your questions. Thanks.