Skip to content
June 4, 2014 / kiranpatils

Sitecore 7.2 Upgrade Learnings

Challenge:

Howdy my dear readers, I am back. Yes, after so long time. But I was learning and exploring new things for you! And please fasten your seat now. As I got a lot of Basics to share with you! So, here we go..

Last week we were working on upgrading from Sitecore.NET 7.0. (rev. 130810) to Sitecore.NET 7.2 (rev. 140228) [You are also going to upgrade to same version? What a co-incidence?]

BTW, Sitecore released 7.2 Update-2 (rev. 140526) on Friday, May 30, 2014 only. So, if you are in planning phase, good to have a look at it here : http://sitecorekh.blogspot.in/2014/05/sitecore-releases-72-update-2-rev-140526.html

After deploying these changes we found following error which took lot of our time. [Roughly 4 hours]

Max-Clause-Count-v2

Message: maxClauseCount is set to 1024
Source: Lucene.Net
Lucene.Net.Search.BooleanQuery.Add(BooleanClause clause)
Lucene.Net.Search.MultiTermQuery.ScoringBooleanQueryRewrite.Rewrite(IndexReader reader, MultiTermQuery query)
Sitecore.ContentSearch.Linq.Lucene.Queries.SpanWildcardQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.Spans.SpanFirstQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.BooleanQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.BooleanQuery.Rewrite(IndexReader reader)
Lucene.Net.Search.IndexSearcher.Rewrite(Query original)

You are also facing same error? Or you would like to be pro-active? Then this post is for you!

Solution:

Just a note : Before we move to solution of this challenge, I would suggest you to check this post — http://maxslabyak.com/sitecore/3-new-sitecore-v7-2-bugs-need-know-go-live/

To resolve this error, we did following steps:

  • We found that — Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config file has following setting (We Increased it from 1024 to 2048, then bit higher and then finally 65536. But it didn’t work):
    <!--  MAX LUCENE QUERY CLAUSE COUNT
            Boolean Max Clause Count. Increasing this value increases memory consumption. Only increase it if you need to run very large queries. 
            This setting allows you to increase or decrease the clause count for Lucene depending upon how big you think the queries could grow.

            Default value: 1024
      -->
      <!-- Performance Issue: Changed value[65536] to default value [1024] -->
      <setting name="ContentSearch.LuceneQueryClauseCount" value="65536" />
  • Then we contacted our common friend — Google and gave us following two links:

http://www.newguid.net/sitecore/2012/sitecore-lucene-max-clause-count/

http://sitec0re.wordpress.com/2014/03/18/lucene-maxclausecount-error-after-upgrading-to-sitecore-7-2-rev-140228/

  • First one, says about pipeline, and second one asks us to do changeinSearchProvider. But we haven’t implemented our search provider — Then we merged both of this solution and derived our solution and here’s the simplified version of it:
    • Create a New Pipeline in your Pipelines projects [Basically, We create layer for each this kind of functionality]. Technically, Create a .cs file — Let’s say AdjustLuceneSettings.cs. And add following code within it:
using Lucene.Net.Search;
using Sitecore.Pipelines;

namespace SitecoreBasics.Pipelines.Search
{    
    public class AdjustLuceneSettings
    {
        ///
        /// Processor to set the lucene max query clause count
        ///
        ///
        public void Process(PipelineArgs args)
        {
            int maxClauseCount = 1024;
            int.TryParse(Sitecore.Configuration.Settings.GetSetting(&quot;ContentSearch.LuceneQueryClauseCount&quot;, &quot;1024&quot;), out maxClauseCount);
            BooleanQuery.MaxClauseCount = maxClauseCount;            
        }       

    }

}
    • Now, this project will need reference of “Lucene.Net.dll” — Please do add it
    • So, Pipeline is ready. Now we need to Hook it in  to Sitecore. To do that, add AdjustLuceneSettings.Config under your App_Config/Include folder, and add following configuration within it:
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <initialize>
        <!--<processor patch:after="*[@type='Sitecore.Pipelines.Loader.EnsureAnonymousUsers, Sitecore.Kernel']" 
                   type="SitecoreBasics.Extension.Pipelines.Search.AdjustLuceneSettings, SitecoreBasics.Extension"/>-->
      </initialize>
    </pipelines>
  </sitecore>
</configuration>
    • Yes, we are near — Now, you can set ContentSearch.LuceneQueryClauseCount to any number you want.
    • That’s it!

So, your application is working fine — right? Happy?! I know you will be. But an engineer within you is shouting and asking following questions:

  1. Why we get maxClauseCount error?
  2. Why ContentSearch.LuceneQueryClauseCount s not working?

We appreciate your questions, and luckily, we’ve answers for both of your questions:

The reason for maxClauseCount is as below:

Recently I ran into a familiar lucene exception within my sitecore solution. Its lucene’s TooManyClauses exception. In fact the exception occures when there are more than 1024 where clauses in your query. That seems a lot but by using a wildcard query for example lucene is translating your wildcard query into a lot of where clauses after analyzing it. Of-course the first step is to take a good look at your query and if possible try to rewrite

Source : http://www.newguid.net/sitecore/2012/sitecore-lucene-max-clause-count/

Answer for guys like me who believe in Basics:

http://dalelane.co.uk/blog/?p=2081

To borrow the example from the Lucene FAQ, in some situations Lucene expands your search query before running it.

If you have an index with:

     car
     cars
     cat
     catalogue
     caterpillar
     delta
     doctor
     dogs
     dominos

And you search for: ca*
It will essentially be expanded to “car OR cars OR cat OR catalogue OR caterpillar” (removing the wildcard).

And if you search for: *
It will be expanded to: “car OR cars OR cat OR catalogue OR caterpillar OR delta OR doctor OR dogs OR dominos“.
A query with nine clauses.

As the index grows, the number of clauses in these wildcard queries increases. If it increases to the point where search queries end up with more than the default 1024 clauses in them, then searches start throwing TooManyClauses exceptions.

And for second question– Why ContentSearch.LuceneQueryClauseCount is not working?

We raised a support case to our second common friends — Sitecore Support and they accepted it as a BUG:

I have managed to replicate this. Unfortunately the “ContentSearch.LuceneQueryClauseCount” setting is not used. We have registered this as a bug for the current version of CMS. We will notify you of any updates. Thank you for reporting the problem.

I hope I’ve answered all your questions. If you have more questions — Just shout! You’ve your set of learning after upgrading? And you would like to share. What a noble thought! Do it right now! Because Sharing is Caring! 🙂

Good reads:

Unable to create document builder – crawling exception after upgrading to SC 7.2 : http://sitecoreblog.marklowe.ch/2014/05/unable-to-create-document-builder/

Happy Upgrading! 🙂

PS : Special thanks to Lilia Silvestrova (Sitecore Support) for providing this suggestion Bhavesh and Marcia who worked on this challenge!

Advertisements

11 Comments

Leave a Comment
  1. Andy / Jun 4 2014 1:22 pm

    Nice tidy fix.

    “Why we get maxClauseCount error?”

    Yes but why suddenly do we get it with the same queries that worked on the previous version? The default is still 1024 right? Presumably due to a lucene update?

    • kiranpatils / Jun 21 2014 3:36 pm

      Thanks — Good question — I also had a same question and asked to Sitecore support guys — They also don’t have a reason. They are going to investigate and find it out for us!

  2. 5 Star Dog Reviews / Jun 4 2014 7:49 pm

    Neat blog! Is your theme custom made or did you download it from somewhere?
    A theme like yours with a few simple tweeks would really make my blog jump out.
    Please let me know where you got your design. Thanks

    • kiranpatils / Jun 21 2014 3:35 pm

      It’s a free theme! — http://theme.wordpress.com/retired/paperpunch/

  3. Sergej Kravcenko / Jun 10 2014 8:45 pm

    Avoid using predecators with StartsWIth, as this converts into wildcard, especially when you check paths, use Contains instead.

    • kiranpatils / Jun 21 2014 3:34 pm

      Sure, Thanks for sharing!

  4. sitecoreottawa / Oct 10 2014 8:14 pm

    Thanks for the post. Was a good reference when I came across this error during an upgrade.

    I noticed that the code for the config file is missing from the post so just wanted to post what I used for others to see. Maybe you can amend the post with your config file when you get a chance.

    • sitecoreottawa / Oct 10 2014 8:15 pm

      -sitecore-
      -pipelines-
      -initialize-
      -processor patch:after=”*[@type=’Sitecore.Pipelines.Loader.EnsureAnonymousUsers, Sitecore.Kernel’]” type=”SitecoreBasics.Pipelines.Search.AdjustLuceneSettings, SitecoreLucenePatch”/-
      -/initialize-
      -/pipelines-
      -/sitecore-
      -/configuration-

    • kiranpatils / Oct 10 2014 11:37 pm

      Thanks for notifying, Updated! — Somehow it got removed

      • kiranpatils / Oct 10 2014 11:39 pm

        Also, Please use Sergej Kravcenko’s Suggestion “Avoid using predecators with StartsWIth, as this converts into wildcard, especially when you check paths, use Contains instead.” — We were having this challenge. And Contains is roughly 40-50% faster than StartsWith — We did that change and reverted it to 1024 default setting!

  5. Alex Tselevich / Aug 25 2015 9:58 pm

    Just a bit of code cleanup, you can condense the whole pipeline Process method into one line:

    BooleanQuery.MaxClauseCount = Settings.GetIntSetting(“ContentSearch.LuceneQueryClauseCount”, 1024);

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: