Challenge:
Howdy my dear readers, I am back. Yes, after so long time. But I was learning and exploring new things for you! And please fasten your seat now. As I got a lot of Basics to share with you! So, here we go..
Last week we were working on upgrading from Sitecore.NET 7.0. (rev. 130810) to Sitecore.NET 7.2 (rev. 140228) – [You are also going to upgrade to same version? What a co-incidence?]
BTW, Sitecore released 7.2 Update-2 (rev. 140526) on Friday, May 30, 2014 only. So, if you are in planning phase, good to have a look at it here : http://sitecorekh.blogspot.in/2014/05/sitecore-releases-72-update-2-rev-140526.html
After deploying these changes we found following error which took lot of our time. [Roughly 4 hours]
Message: maxClauseCount is set to 1024
Source: Lucene.Net
Lucene.Net.Search.BooleanQuery.Add(BooleanClause clause)
Lucene.Net.Search.MultiTermQuery.ScoringBooleanQueryRewrite.Rewrite(IndexReader reader, MultiTermQuery query)
Sitecore.ContentSearch.Linq.Lucene.Queries.SpanWildcardQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.Spans.SpanFirstQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.BooleanQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.BooleanQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.IndexSearcher.Rewrite(Query original)
You are also facing same error? Or you would like to be pro-active? Then this post is for you!
Solution:
Just a note : Before we move to solution of this challenge, I would suggest you to check this post — http://maxslabyak.com/sitecore/3-new-sitecore-v7-2-bugs-need-know-go-live/
To resolve this error, we did following steps:
- We found that — Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config file has following setting (We Increased it from 1024 to 2048, then bit higher and then finally 65536. But it didn’t work):
<!-- MAX LUCENE QUERY CLAUSE COUNT
Boolean Max Clause Count. Increasing this value increases memory consumption. Only increase it if you need to run very large queries.
This setting allows you to increase or decrease the clause count for Lucene depending upon how big you think the queries could grow.
Default value: 1024
-->
<!-- Performance Issue: Changed value[65536] to default value [1024] -->
<setting name="ContentSearch.LuceneQueryClauseCount" value="65536" />
- Then we contacted our common friend — Google and gave us following two links:
http://www.newguid.net/sitecore/2012/sitecore-lucene-max-clause-count/
Lucene MaxClauseCount error after upgrading to Sitecore 7.2 (rev. 140228)
- First one, says about pipeline, and second one asks us to do changeinSearchProvider. But we haven’t implemented our search provider — Then we merged both of this solution and derived our solution and here’s the simplified version of it:
- Create a New Pipeline in your Pipelines projects [Basically, We create layer for each this kind of functionality]. Technically, Create a .cs file — Let’s say AdjustLuceneSettings.cs. And add following code within it:
using Lucene.Net.Search;
using Sitecore.Pipelines;
namespace SitecoreBasics.Pipelines.Search
{
public class AdjustLuceneSettings
{
///
/// Processor to set the lucene max query clause count
///
///
public void Process(PipelineArgs args)
{
int maxClauseCount = 1024;
int.TryParse(Sitecore.Configuration.Settings.GetSetting("ContentSearch.LuceneQueryClauseCount", "1024"), out maxClauseCount);
BooleanQuery.MaxClauseCount = maxClauseCount;
}
}
}
- Now, this project will need reference of “Lucene.Net.dll” — Please do add it
- So, Pipeline is ready. Now we need to Hook it in to Sitecore. To do that, add AdjustLuceneSettings.Config under your App_Config/Include folder, and add following configuration within it:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
<sitecore>
<pipelines>
<initialize>
<!--<processor patch:after="*[@type='Sitecore.Pipelines.Loader.EnsureAnonymousUsers, Sitecore.Kernel']"
type="SitecoreBasics.Extension.Pipelines.Search.AdjustLuceneSettings, SitecoreBasics.Extension"/>-->
</initialize>
</pipelines>
</sitecore>
</configuration>
- Yes, we are near — Now, you can set ContentSearch.LuceneQueryClauseCount to any number you want.
- That’s it!
So, your application is working fine — right? Happy?! I know you will be. But an engineer within you is shouting and asking following questions:
- Why we get maxClauseCount error?
- Why ContentSearch.LuceneQueryClauseCount s not working?
We appreciate your questions, and luckily, we’ve answers for both of your questions:
The reason for maxClauseCount is as below:
Recently I ran into a familiar lucene exception within my sitecore solution. Its lucene’s TooManyClauses exception. In fact the exception occures when there are more than 1024 where clauses in your query. That seems a lot but by using a wildcard query for example lucene is translating your wildcard query into a lot of where clauses after analyzing it. Of-course the first step is to take a good look at your query and if possible try to rewrite
Source : http://www.newguid.net/sitecore/2012/sitecore-lucene-max-clause-count/
Answer for guys like me who believe in Basics:
http://dalelane.co.uk/blog/?p=2081
To borrow the example from the Lucene FAQ, in some situations Lucene expands your search query before running it.
If you have an index with:
car
cars
cat
catalogue
caterpillar
delta
doctor
dogs
dominos
And you search for: ca*
It will essentially be expanded to “car OR cars OR cat OR catalogue OR caterpillar” (removing the wildcard).
And if you search for: *
It will be expanded to: “car OR cars OR cat OR catalogue OR caterpillar OR delta OR doctor OR dogs OR dominos“.
A query with nine clauses.
As the index grows, the number of clauses in these wildcard queries increases. If it increases to the point where search queries end up with more than the default 1024 clauses in them, then searches start throwing TooManyClauses exceptions.
And for second question– Why ContentSearch.LuceneQueryClauseCount is not working?
We raised a support case to our second common friends — Sitecore Support and they accepted it as a BUG:
I have managed to replicate this. Unfortunately the “ContentSearch.LuceneQueryClauseCount” setting is not used. We have registered this as a bug for the current version of CMS. We will notify you of any updates. Thank you for reporting the problem.
I hope I’ve answered all your questions. If you have more questions — Just shout! You’ve your set of learning after upgrading? And you would like to share. What a noble thought! Do it right now! Because Sharing is Caring! 🙂
Good reads:
Unable to create document builder – crawling exception after upgrading to SC 7.2 :
Unable to create document builder – crawling exception after upgrading to SC 7.2
Happy Upgrading! 🙂
PS : Special thanks to Lilia Silvestrova (Sitecore Support) for providing this suggestion Bhavesh and Marcia who worked on this challenge!
It contains following directories and files:
Sitecore basics! 