Elasticsearch Filter And Query

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Elasticsearch Filter And Query

shanmuthu83
Hi All

Good Morning to All

In elastic search query have Filter and Query..

Which first execute filter or query..


--
Regards
 
Shanmugam.M
 

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

dadoonet
AFAIK Filter first reduces the number of docs to query on.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 4 janv. 2013 à 07:48, Shanmugam Muthanandham <[hidden email]> a écrit :

Hi All

Good Morning to All

In elastic search query have Filter and Query..

Which first execute filter or query..


--
Regards
 
Shanmugam.M
 

--
 
 

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

shanmuthu83
Okie Thanks David

On Fri, Jan 4, 2013 at 12:32 PM, David Pilato <[hidden email]> wrote:
AFAIK



--
Regards
 
Shanmugam.M
 

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

Igor Motov-3
I would say it depends. If you are asking about top level filter and top level query then query is executed first and then results are filtered. 

On Friday, January 4, 2013 2:25:06 AM UTC-5, shanmugam wrote:
Okie Thanks David

On Fri, Jan 4, 2013 at 12:32 PM, David Pilato <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="YiIf0fZHjDgJ">da...@...> wrote:
AFAIK



--
Regards
 
Shanmugam.M
 

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

dadoonet
Thanks Igor for correcting my mistake.
So, in which case filter is applied before the query?

Le 4 janvier 2013 à 14:24, Igor Motov <[hidden email]> a écrit :
I would say it depends. If you are asking about top level filter and top level querythen query is executed first and then results are filtered. 

On Friday, January 4, 2013 2:25:06 AM UTC-5, shanmugam wrote:
Okie Thanks David

On Fri, Jan 4, 2013 at 12:32 PM, David Pilato <[hidden email]> wrote:
AFAIK


 
--
 
Regards
 
Shanmugam.M
 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

Igor Motov-3
In 0.20.x and below, it's probably valid to say that filter is applied before the query in the filtered query. Technically, both query and filter documents are advanced together, but since filter is consulted first, it can short-circuit the query execution. In 0.21.x it's more complicated and depends on types of queries and filters and can be specified on the query. 

On Friday, January 4, 2013 8:46:33 AM UTC-5, David Pilato wrote:
Thanks Igor for correcting my mistake.
So, in which case filter is applied before the query?

Le 4 janvier 2013 à 14:24, Igor Motov <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="muDuVN5_GigJ">imo...@...> a écrit :
I would say it depends. If you are asking about top level filter and top level querythen query is executed first and then results are filtered. 

On Friday, January 4, 2013 2:25:06 AM UTC-5, shanmugam wrote:
Okie Thanks David

On Fri, Jan 4, 2013 at 12:32 PM, David Pilato <[hidden email]> wrote:
AFAIK


 
--
 
Regards
 
Shanmugam.M
 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

dadoonet
Thanks Igor!

Le 4 janvier 2013 à 16:32, Igor Motov <[hidden email]> a écrit :
In 0.20.x and below, it's probably valid to say that filter is applied before the query in the filtered query. Technically, both query and filter documents are advanced together, but since filter is consulted first, it can short-circuit the query execution. In 0.21.x it's more complicated and depends on types of queries and filters and can be  specified on the query. 
 

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

phill
In reply to this post by Igor Motov-3
I assume by "short circuit" you mean that if a filter results in NO matches then the query is skipped.
Otherwise, how does running the filter first help in a scenerio where a filter might reduce the total documents to 1% of all documents?  My user view was that two "bitsets"of matches: one from the query & one from the filter where "and"ed together, so what matter would  it make to do one or the other 1st?
Apparently, such a simplistic view is incorrect.  I'm interested to know how to make filters (which might be cached) feed into a query, so the query doesn't bother with as many docs and only processes that with has been pre-filtered.

-Paul

On 1/4/2013 7:32 AM, Igor Motov wrote:
In 0.20.x and below, it's probably valid to say that filter is applied before the query in the filtered query. Technically, both query and filter documents are advanced together, but since filter is consulted first, it can short-circuit the query execution. In 0.21.x it's more complicated and depends on types of queries and filters and can be specified on the query. 


--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

Igor Motov-3
Paul,

I think your "simplistic" view is correct in some cases. But there are also cases where "bitsets" don't exist. Consider this somewhat absurd query for example: https://gist.github.com/111db5810fadcbf5f4c8


On Saturday, January 5, 2013 4:06:39 AM UTC-5, P Hill wrote:
I assume by "short circuit" you mean that if a filter results in NO matches then the query is skipped.
Otherwise, how does running the filter first help in a scenerio where a filter might reduce the total documents to 1% of all documents?  My user view was that two "bitsets"of matches: one from the query & one from the filter where "and"ed together, so what matter would  it make to do one or the other 1st?
Apparently, such a simplistic view is incorrect.  I'm interested to know how to make filters (which might be cached) feed into a query, so the query doesn't bother with as many docs and only processes that with has been pre-filtered.

-Paul

On 1/4/2013 7:32 AM, Igor Motov wrote:
In 0.20.x and below, it's probably valid to say that filter is applied before the query in the filtered query. Technically, both query and filter documents are advanced together, but since filter is consulted first, it can short-circuit the query execution. In 0.21.x it's more complicated and depends on types of queries and filters and can be specified on the query. 


--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch Filter And Query

phill
My mental model does help me to see why this is a special case.

you inserted 10 docs each with a doc field 1 .. 10.
The filter finds 5, 6 AND 7; The query only matches 5 OR 7.
So why would each NOT be represented as a bitset (a compact list of docs that match each represented in order by a yes or no)?  Is it because of the constant score?  I'm not seeing it.

-Paul

On 1/7/2013 7:11 PM, Igor Motov wrote:
Paul,

I think your "simplistic" view is correct in some cases. But there are also cases where "bitsets" don't exist. Consider this somewhat absurd query for example: https://gist.github.com/111db5810fadcbf5f4c8


On Saturday, January 5, 2013 4:06:39 AM UTC-5, P Hill wrote:
I assume by "short circuit" you mean that if a filter results in NO matches then the query is skipped.
Otherwise, how does running the filter first help in a scenerio where a filter might reduce the total documents to 1% of all documents?  My user view was that two "bitsets"of matches: one from the query & one from the filter where "and"ed together, so what matter would  it make to do one or the other 1st?
Apparently, such a simplistic view is incorrect.  I'm interested to know how to make filters (which might be cached) feed into a query, so the query doesn't bother with as many docs and only processes that with has been pre-filtered.

-Paul

On 1/4/2013 7:32 AM, Igor Motov wrote:
In 0.20.x and below, it's probably valid to say that filter is applied before the query in the filtered query. Technically, both query and filter documents are advanced together, but since filter is consulted first, it can short-circuit the query execution. In 0.21.x it's more complicated and depends on types of queries and filters and can be specified on the query. 


--
 
 

--