SOLR Filtering Performance Increase

A couple months ago I wrote about the terrible performance and a work around for SOLR / Lucene search engine. I discovered that performance would drop off a cliff while using filter queries to narrow search results for search queries on common terms in large indexes.  Although, it looks like the issue has been addressed in some of the latest nightly SOLR builds and is scheduled for official release with SOLR v1.4. Previous to this new version the filter queries were applied after the main query ran. This is all well and good but it doesn’t help speed your query up like you think it should. The new version applies the filters in parallel to the main query significantly speeding up searches with common queries and query filters by 30% to 80% along with a 40% smaller memory footprint.

However, even with this speed improvement you still should consider how you structure your queries. There is no need to do a query across every field if you know you really want to filter everything down with a single filter query. Try moving that filter query (fq) into the actual query (q) as <field>:<filter>. You might be surprised by the results…