index.query.bool.max_clause_count Setting and TermsQueryParser

[jrawlings]

In issue #482 (https://github.com/elasticsearch/elasticsearch/issues/
482), Shay mentioned to set the property
'index.query.bool.max_clause_count' in order to raise the max clause
count for a lucene bool query. I've tried to set this property but it
doesn't seem to be persisted. Am I doing something wrong? Here's what
I'm doing: https://gist.github.com/1019762

When I do get around to setting this property, will it fix a Lucene
TooManyClauses error w/ the TermsQueryParser if I have over 1024
terms? I noticed that the setting is applied for the the
BoolQueryParser only from that issue (482).

Any help or thoughts would be greatly appreciated.

Thanks,
JR

[Stéphane Raux]

You should provide the property in the following way :
 curl -XPUT "http://localhost:9200/_settings" -d '

>  {
>   "index" : {
>      "query" : {
>          "bool" : {
>             "max_clause_count": 2048
>            }
>       }
>     }
>  }
> '

Best,

Stéphane

2011/6/10 Joe Rawlings <[hidden email]>:

> In issue #482 (https://github.com/elasticsearch/elasticsearch/issues/
> 482), Shay mentioned to set the property
> 'index.query.bool.max_clause_count' in order to raise the max clause
> count for a lucene bool query. I've tried to set this property but it
> doesn't seem to be persisted. Am I doing something wrong? Here's what
> I'm doing: https://gist.github.com/1019762
>
> When I do get around to setting this property, will it fix a Lucene
> TooManyClauses error w/ the TermsQueryParser if I have over 1024
> terms? I noticed that the setting is applied for the the
> BoolQueryParser only from that issue (482).
>
> Any help or thoughts would be greatly appreciated.
>
> Thanks,
> JR
>
>
>

[jrawlings]

That's essentially the same thing I was doing. Still, I tried it and
the property didn't show up when I retrieved the settings. :/

On Jun 11, 12:51 am, Stéphane Raux <[hidden email]> wrote:

> You should provide the property in the following way :
>  curl -XPUT "http://localhost:9200/_settings" -d '
>
> >  {
> >   "index" : {
> >      "query" : {
> >          "bool" : {
> >             "max_clause_count": 2048
> >            }
> >       }
> >     }
> >  }
> > '
>
> Best,
>
> Stéphane
>
> 2011/6/10 Joe Rawlings <[hidden email]>:
>
>
>
>
>
>
>
> > In issue #482 (https://github.com/elasticsearch/elasticsearch/issues/
> > 482), Shay mentioned to set the property
> > 'index.query.bool.max_clause_count' in order to raise the max clause
> > count for a lucene bool query. I've tried to set this property but it
> > doesn't seem to be persisted. Am I doing something wrong? Here's what
> > I'm doing:https://gist.github.com/1019762
>
> > When I do get around to setting this property, will it fix a Lucene
> > TooManyClauses error w/ the TermsQueryParser if I have over 1024
> > terms? I noticed that the setting is applied for the the
> > BoolQueryParser only from that issue (482).
>
> > Any help or thoughts would be greatly appreciated.
>
> > Thanks,
> > JR

[dadoonet]

I think that you should remove your index before sending settings.

Try this :
curl -XDELETE "http://localhost:9200/myindex" -d ' {}'

curl -XPUT "http://localhost:9200/myindex/_settings" -d '
 {
  "index" : {
     "query" : {
         "bool" : {
            "max_clause_count": 2048
           }
      }
    }
 }
'

Hope this helps.
David ;-)

Le 11 juin 2011 à 16:30, Joe Rawlings <[hidden email]> a écrit :

> That's essentially the same thing I was doing. Still, I tried it and
> the property didn't show up when I retrieved the settings. :/
>
> On Jun 11, 12:51 am, Stéphane Raux <[hidden email]> wrote:
>> You should provide the property in the following way :
>>  curl -XPUT "http://localhost:9200/_settings" -d '
>>
>>>  {
>>>   "index" : {
>>>      "query" : {
>>>          "bool" : {
>>>             "max_clause_count": 2048
>>>            }
>>>       }
>>>     }
>>>  }
>>> '
>>
>> Best,
>>
>> Stéphane
>>
>> 2011/6/10 Joe Rawlings <[hidden email]>:
>>
>>
>>
>>
>>
>>
>>
>>> In issue #482 (https://github.com/elasticsearch/elasticsearch/issues/
>>> 482), Shay mentioned to set the property
>>> 'index.query.bool.max_clause_count' in order to raise the max clause
>>> count for a lucene bool query. I've tried to set this property but it
>>> doesn't seem to be persisted. Am I doing something wrong? Here's what
>>> I'm doing:https://gist.github.com/1019762
>>
>>> When I do get around to setting this property, will it fix a Lucene
>>> TooManyClauses error w/ the TermsQueryParser if I have over 1024
>>> terms? I noticed that the setting is applied for the the
>>> BoolQueryParser only from that issue (482).
>>
>>> Any help or thoughts would be greatly appreciated.
>>
>>> Thanks,
>>> JR

[kimchy]

You can't set this setting "on the fly", since it gets applied when an index gets created.

On Sunday, June 12, 2011 at 9:05 AM, David Pilato wrote:

I think that you should remove your index before sending settings.

Try this :
curl -XDELETE "http://localhost:9200/myindex" -d ' {}'

curl -XPUT "http://localhost:9200/myindex/_settings" -d '
{
"index" : {
"query" : {
""bool" : {
"max_clause_count": 2048
}}
}
}
}
'

Hope this helps.
David ;-)

Le 11 juin 2011 à 16:30, Joe Rawlings <[hidden email]> a écrit :

That's essentially the same thing I was doing. Still, I tried it and
the property didn't show up when I retrieved the settings. :/

On Jun 11, 12:51 am, Stéphane Raux <[e-mail]> wrote:

You should provide the property in the following way :
curl -XPUT "http://localhost:9200/_settings" -d '

{
"index" : {
"query" : {
"bool" : {
"max_clause_count": 2048
}
}
}
}
'

Best,

Stéphane

2011/6/10 Joe Rawlings <[e-mail]>:

In issue #482 (https://github.com/elasticsearch/elasticsearch/issues/
482), Shay mentioned to set the property
'index.query.bool.max_clause_count' in order to raise the max clause
count for a lucene bool query. I've tried to set this property but it
doesn't seem to be persisted. Am I doing something wrong? Here's what
I'm doing:https://gist.github.com/1019762

When I do get around to setting this property, will it fix a Lucene
TooManyClauses error w/ the TermsQueryParser if I have over 1024
terms? I noticed that the setting is applied for the the
BoolQueryParser only from that issue (482).

Any help or thoughts would be greatly appreciated.

Thanks,
JR

[jrawlings]

Cool, so I started reindexing my data with the
"index.query.bool.max_clause_count" set to 8192 and retried the query
that was causing the issue before. The same 'maxClauseCount is set to
1024' error showed up again. I put a breakpoint in the BoolQueryParser
and reran the query but nothing was hit. Any thoughts?

On Jun 10, 2:03 pm, Joe Rawlings <[hidden email]> wrote:
> When I do get around to setting this property, will it fix a Lucene
> TooManyClauses error w/ the TermsQueryParser if I have over 1024
> terms? I noticed that the setting is applied for the the
> BoolQueryParser only from that issue (482).

[kimchy]

You don't need to reindex your data for that... . You could have closed and reopened your index for example. Or just set it in the elasticsearch.yml file. This setting is tricky, since its a static one (Lucene wise). So, I suggest setting it in the elasticsearch.yml file so it will always apply.

On Monday, June 13, 2011 at 7:37 PM, Joe Rawlings wrote:

Cool, so I started reindexing my data with the
"index.query.bool.max_clause_count" set to 8192 and retried the query
that was causing the issue before. The same 'maxClauseCount is set to
1024' error showed up again. I put a breakpoint in the BoolQueryParser
and reran the query but nothing was hit. Any thoughts?

On Jun 10, 2:03 pm, Joe Rawlings <[e-mail]> wrote:

When I do get around to setting this property, will it fix a Lucene
TooManyClauses error w/ the TermsQueryParser if I have over 1024
terms? I noticed that the setting is applied for the the
BoolQueryParser only from that issue (482).

[jrawlings]

Perfect, thank you

On Jun 13, 10:12 am, Shay Banon <[hidden email]> wrote:

> You don't need to reindex your data for that... . You could have closed and reopened your index for example. Or just set it in the elasticsearch.yml file. This setting is tricky, since its a static one (Lucene wise). So, I suggest setting it in the elasticsearch.yml file so it will always apply.
>
>
>
>
>
>
>
> On Monday, June 13, 2011 at 7:37 PM, Joe Rawlings wrote:
> > Cool, so I started reindexing my data with the
> > "index.query.bool.max_clause_count" set to 8192 and retried the query
> > that was causing the issue before. The same 'maxClauseCount is set to
> > 1024' error showed up again. I put a breakpoint in the BoolQueryParser
> > and reran the query but nothing was hit. Any thoughts?
>
> > On Jun 10, 2:03 pm, Joe Rawlings <[hidden email] (http://gmail.com)> wrote:
> > > When I do get around to setting this property, will it fix a Lucene
> > > TooManyClauses error w/ the TermsQueryParser if I have over 1024
> > > terms? I noticed that the setting is applied for the the
> > > BoolQueryParser only from that issue (482).

[pranmitt]

Did you face any performance impact when you tried to query it for more than 1k terms ?