Comprehensive data protection for all workloads
Post Reply
JohnG
Novice
Posts: 8
Liked: never
Joined: Aug 26, 2010 1:23 pm
Contact:

VBRCatalog folder growing in size

Post by JohnG »

I installed Veeam a few days ago and it seems that my VBRCatalog is growing indiscriminately. I have added 15 gigs of space over the past 3 days to this disk (Veeam is running in a VM) and it is already full today. Since the VM runs out of space, jobs fail.

1) What does this folder do exactly?

2) Can I limit the size of it? (ie, can I periodically delete its contents?) What are some potential reasons for it growing so much? I have 4 jobs that run once an hour...

3) How can I move it to another folder/disk so I don't have to constantly grow the system disk? Short of reinstalling Veeam.

Thanks.

Edit: The 'index\Machines' subfolder is the one that seems to hold most of the data.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

The folder contains guest file system index, you have probably missed the setup wizard step explaining this (there was an option to set folder location different than default). Each restore point (and you are backing up every hour) produces its own index file. You may want to enable indexing for only certain guests, for which you really need it. You can refer to sticky FAQ topic on top of this forum for more info on guest file system indexing.

Yes, you should be able to move the folder by updating the corresponding registry key. I have not personally tested this though.

Thanks!

JohnG
Novice
Posts: 8
Liked: never
Joined: Aug 26, 2010 1:23 pm
Contact:

Re: VBRCatalog folder growing in size

Post by JohnG »

I've decided to disable guest indexing completely on my backups (and deleted whatever indexes existed). If I ever do a restore, it will be a full VM and not individual guest files. Plus I have DPM backing up my actual file server data so I would use that to restore in such a situation.

Thanks.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

Actually, indexing is not required to restore individual files (see FAQ). You can restore individual files whether or not you perform guest file system indexing.

bjamieson
Lurker
Posts: 2
Liked: never
Joined: Nov 11, 2010 7:08 pm
Full Name: Bruce Jamieson

Re: VBRCatalog folder growing in size

Post by bjamieson »

So... has an answer been found? I looked thru the registry and there are 5 or 6 references to the VBRCatalog. If I change the 2 keys pointing to the path, what about the other references? There must be an easier way to move the catalog?
Cheers

Vitaliy S.
Product Manager
Posts: 24246
Liked: 1859 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VBRCatalog folder growing in size

Post by Vitaliy S. »

Bruce, could you please contact our technical team for assistance with this? I guess changing registry keys is not enough to move a catalog folder.

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Is there any way to predict / estimate disk space growth for the VBRCatalog folder?

In my example 7 day backup for 250 VMs. Currently have 40GB drive allocated for the folder and filling it up quickly. I want to know if it makes sense to add more disk space or to limit which VMs get indexed or maybe stop indexing them completely.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

Are you asking about Backup server, or about Enterprise Manager server?

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Gostev wrote:Are you asking about Backup server, or about Enterprise Manager server?
I'm asking about Enterprise Manager server.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

For Enterprise Manager, you multiple that by your catalog retention policy (which can be adjusted in the Enterprise Manager configuration). For example, if it is 3 months, then 40GBx4x3 = about 500GB. Thanks.

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Ok thanks. So I'm looking at the Catalog Settings in enterprise manager. Retention period is set to 60 months. I'd love to change it but the option is greyed out. Can't seem to find any information on this in the user guide.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

That means you are using Standard Edition, which is limited to keeping catalog on for backup files residing on disk. So, if you only keep 7 days worth of restore points on disk, the catalog will only keep data for these 7 days and should not grow further. Thanks.

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Ok so I confused the issue. I do see that the Index subfolder under VBRCatalog does have 7 days worth and that's fine. The folder where I'm seeing growth is the Journal subfolder. I installed enterprise manager in May and I am seeing 3 subfolders under Journal: 2011_05, 2011_07 and 2011_08. These are the large folders. For example 2011_07 is 15GB and 2011_08, newest one which was just created a few days ago is already 5GB. Is there more details about these folders and how to go about managing their size?

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

Journal folder contains contents presented to Search Server for indexing (text files). Amount of folders should be according to your catalog retention policy, as I explained above. Thanks!

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Ok so then it should be ok to delete the folder from May? Shouldn't it have been cleaned up automatically since I am only doing 7 days retention?

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

Job retention (7 days) has nothing to deal with this folder. Reading through my responses above, the data in this folder will be kept for the amount of months as specified in the Enterprise Manager, under Catalog settings. You may want to reduce that amount to make sure older data is deleted automatically. Thanks.

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

Yes and I replied that it is set to 60 months but greyed out on my end. To which you replied that is because I am running standard version of EM. So how do I go about reducing the 60 month setting to 1 month? Thanks.

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

So, which Veeam Backup edition do you have in reality? Is it Standard? Then it should not keep this history at all. I may need to talk to the devs about this.

greendx
Influencer
Posts: 13
Liked: never
Joined: May 18, 2011 2:06 pm
Full Name: Eugene Lipsky
Location: New York
Contact:

Re: VBRCatalog folder growing in size

Post by greendx »

I have the standard version. Let me know if I should open a formal support ticket for this.

Thanks
Eugene

gccit
Novice
Posts: 5
Liked: never
Joined: Mar 26, 2010 6:15 pm
Full Name: GCC
Contact:

Re: VBRCatalog folder growing in size

Post by gccit »

Found this post today while searching for info about my growing VBRcatalog\Index folder. We're working towards using Veeam as our ONLY backup product - eliminating Commvault and DPM2010. To do so, it's requiring much longer Veeam retention periods. 360 days or more in some instances. I love having indexing available when I'm looking for a specific file, but the disk space cost for the index long-term is too much really. I'd love if there was a way to delete indexes after a given # of days.

On a related note, if I disable indexing for a job that has previously had indexing enabled, will it purge them during the next backup run?

Gostev
SVP, Product Management
Posts: 26693
Liked: 4273 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBRCatalog folder growing in size

Post by Gostev »

v6 should reduce the size of the index in about 5 times.
No, job setting has no effect on already created catalog data.

itdirector
Enthusiast
Posts: 59
Liked: 3 times
Joined: Jan 19, 2012 8:53 pm
Full Name: friedman

[MERGED] Large Index directory

Post by itdirector »

VBR 6.1
vSphere 5.0
Daily backup jobs - 30 day retension
Monthly backup jobs - 36 month retension

We have been running VBR for about 9 months now in 5 offices (5 VBR installations). In one of our offices (main office), the index folder (only 25 VMs) has grown to 95GB!
"D:\Program Files\Veeam\Backup and Replication\VBRCatalog\Index\Machines\"

- What is the correct procedure to move these existing indexes (the entire folder & GuestIndexData.zip files) to a larger drive (file copy + registry change?)
- What happens if we disable indexing for the jobs? Does that mean enterprise manager won't be able to search for individual files in backed up VMs anymore?

Thx

Vitaliy S.
Product Manager
Posts: 24246
Liked: 1859 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VBRCatalog folder growing in size

Post by Vitaliy S. »

1. Please see this KB article for the detailed procedure on how to accomplish this > http://cp.veeam.com/knowledgebase/artic ... ormat/html
2. EM will be able to search through the existing restore points that are already saved in the Catalog folder. All restore points that are done with VM indexing disabled, will not be displayed in the search results.

itdirector
Enthusiast
Posts: 59
Liked: 3 times
Joined: Jan 19, 2012 8:53 pm
Full Name: friedman

Re: VBRCatalog folder growing in size

Post by itdirector »

Thx Vitaliy. That article states it is for VBR 5.x ; is it still valid for 6.1 & 6.5?

Vitaliy S.
Product Manager
Posts: 24246
Liked: 1859 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VBRCatalog folder growing in size

Post by Vitaliy S. »

Yes, it should!

Andy
Lurker
Posts: 1
Liked: never
Joined: Oct 08, 2013 12:05 am
Full Name: Quoc Anh
Contact:

Re: VBRCatalog folder growing in size

Post by Andy »

Hi,

Please correct me if I'm wrong,
We running VEEAM B&R 7.0, I found there is 2 places stocking .idf files

1-C:\VBRCatalog
=> I believe this is for "Enable Guest file system indexing" in a backup job
which you can move by moving the folder to another storage and change the registry key HLM\SOFTWARE\Veeam\Veeam Backup Catalog\CatalogPath to reflect the physical location.

2-C:\Users\svc_operator\AppData\Local\Temp
=> this one is I believe for the "Gest file system catalog" setting on VEEAM Backup Enterprise Manager

In my observation, both of them taking a lot of space, sure depending on how long do you set the parameters.

I have a question, how can we move the 2rd catalog ? the one store/log "Gest file system catalog" ?

Many thanks.

foggy
Veeam Software
Posts: 19444
Liked: 1765 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VBRCatalog folder growing in size

Post by foggy »

Andy, please take a look at this post, should clearly answer your questions. Thanks.

Post Reply

Who is online

Users browsing this forum: Ash_P, Bing [Bot], Google [Bot] and 71 guests