-
- VP, Product Management
- Posts: 6032
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
VBRCatalog with NTFS Compression
Any reason not to enable NTFS compression on the VBRCatalog folder? I ask because our catalog has already grown to >20GB and it appears to contain mostly text file listings that compress very well. A quick test gave a 3:1 compression ratio which is a pretty big savings on the Veeam server.
-
- Chief Product Officer
- Posts: 31754
- Liked: 7259 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: VBRCatalog with NTFS Compression
This might be a good idea. I will run this by developers.
-
- Chief Product Officer
- Posts: 31754
- Liked: 7259 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: VBRCatalog with NTFS Compression
OK, so devs think this might be good idea, only worried if Search Server can still crawl such folder (pretty sure there will be no problems, but this was never explicitly tested).
-
- VP, Product Management
- Posts: 6032
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: VBRCatalog with NTFS Compression
I'm pretty sure it will still crawl it with no issues but I'll have to admit that we're not even using the search server at this time, just the tree view.
-
- Chief Product Officer
- Posts: 31754
- Liked: 7259 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: VBRCatalog with NTFS Compression
...which would not even be available in v5, if not your feedback during early beta funny how things work out sometimes.
-
- Enthusiast
- Posts: 36
- Liked: 5 times
- Joined: Jan 04, 2010 4:14 pm
- Full Name: Stan
- Contact:
Re: VBRCatalog with NTFS Compression
How did this turn out?
Thanks,
Stan
Thanks,
Stan
-
- VP, Product Management
- Posts: 6032
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: VBRCatalog with NTFS Compression
Seems to be working great for me and save a huge amount of space. I've experienced no issues.
-
- Enthusiast
- Posts: 27
- Liked: 1 time
- Joined: Sep 06, 2011 5:57 am
- Full Name: Josh
- Contact:
Re: VBRCatalog with NTFS Compression
Has anyone tried this in v6? Have seen Gostev note that these should be 5x smaller in v6, but my experience to date is these are now 5x larger..
-
- Veteran
- Posts: 261
- Liked: 29 times
- Joined: May 03, 2011 12:51 pm
- Full Name: James Pearce
- Contact:
Re: VBRCatalog with NTFS Compression
Have to say, I've long been a fan of NTFS compression and have used it wherever I can since P-Pro & NT days. For example file shares (less important now with docx etc being internally zipped), IIS log directories and so on are all easy targets. Technically the system works under the file cache, so cached content has no impact (rather the lazy writer and reads incur the penalty). I once read that the overhead was something like 30 clocks per byte on a P-Pro, which seemed to tally with throughput back then.
-
- Enthusiast
- Posts: 27
- Liked: 1 time
- Joined: Sep 06, 2011 5:57 am
- Full Name: Josh
- Contact:
Re: VBRCatalog with NTFS Compression
Nice one.. this definately sounds like a go-er then - will log a change request to enable compression and let you know how this goes for us with v6.
-
- Enthusiast
- Posts: 82
- Liked: 33 times
- Joined: Mar 25, 2013 7:37 pm
- Full Name: Lars Pisanec
- Contact:
[MERGED] Compress B&R catalog folder
Hi!
Are there any downsides to use NTFS compression on the B&R catalog folder?
Compression works well on the *.ifd files, most of the time 50% or more.
Kind regards,
Lars
Are there any downsides to use NTFS compression on the B&R catalog folder?
Compression works well on the *.ifd files, most of the time 50% or more.
Kind regards,
Lars
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: VBRCatalog with NTFS Compression
Lars, enabling NTFS compression on the VBRCatalog folder is a pretty common practice among our customers.
Also, probably the following topics will be useful:
cleaning up vbrcatalog folder
VBRCatalog folder growing in size
Also, probably the following topics will be useful:
cleaning up vbrcatalog folder
VBRCatalog folder growing in size
-
- Enthusiast
- Posts: 82
- Liked: 33 times
- Joined: Mar 25, 2013 7:37 pm
- Full Name: Lars Pisanec
- Contact:
Re: VBRCatalog with NTFS Compression
Thanks foggy!
I did a brief search on compression and this (now merged) thread did not show up. Ah well the wonders of choosing the right terms to search for never cease to amaze
I did a brief search on compression and this (now merged) thread did not show up. Ah well the wonders of choosing the right terms to search for never cease to amaze
-
- Chief Product Officer
- Posts: 31754
- Liked: 7259 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: VBRCatalog with NTFS Compression
No worries, Lars. This is exactly the reason why we merge topics - this adds more search terms to the topic due to adding additional ways of asking the same question. Thanks for trying the search before you post.
-
- Expert
- Posts: 227
- Liked: 62 times
- Joined: Apr 10, 2014 4:13 pm
- Contact:
Re: VBRCatalog with NTFS Compression
Has anyone tried Windows Deduplication on the catalog?
-
- Chief Product Officer
- Posts: 31754
- Liked: 7259 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: VBRCatalog with NTFS Compression
Yes and it's awesome. In fact, I recommended it in one of the weekly forum digests earlier this yearlightsout wrote:Has anyone tried Windows Deduplication on the catalog?
gostev wrote:And to finish off for today, I wanted to discuss our integration with Microsoft Search Server (MSS), as it has been a subject to many questions lately. B&R have had MSS integration for over 5 years ago, back when we have introduced guest file system indexing in v4. Search was needed to enable our users to quickly find files in large catalogs of thousands of VMs, allowing for fast file level recovery when the location of the lost file was not known. Using MSS has always been optional, as we have also provided a built-in search engine that is used until you register at least one MSS with the Enterprise Manager. Since the built-in engine only scaled for up to a few hundreds of VMs, leveraging MSS integration was very popular at the time.
But, fast forward to 2015 – and a lot has changed since then! Over the past years, built-in search engine performance has been improved significantly, both due to optimizations in the code, and much faster storage hardware. On the other hand, through our support we learned that MSS falls apart performing searches against very large catalogs - while the built-in engine is able to query catalogs of virtually any size reliably (albeit not as fast). As the result, these days I don't know of many clients using MSS integration, and I do not recommend it anymore. We are also actively looking to replace it with the different search engine down the road.
But for now, best results can be achieved by optimizing built-in search engine performance, and for that I recommend simply placing the catalog on an SSD drive. And here we run into one associated issue that large customers often complain about: the disk space needed for the catalog. But, this one can be easily solved too! One of our Solution Architects has found that placing catalog on a Windows Server dedupe volume works very well and makes search even faster in some scenarios, likely because the number of total blocks that must be read from disk is significantly reduced. Bottom line, our current recommendation for customers looking to index and search a very large amount of VMs, is to: use built-in search engine, place the catalog on a fast, dedicated volume (preferably backed by an SSD drive), and enable Windows Server dedupe on that volume.
Who is online
Users browsing this forum: Google [Bot] and 52 guests