Quantcast

Facets have got me down

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Facets have got me down

Justin Treher
It might be because I'm low on energy, but I'm having a real tough time reconciling facet structure in my head right now. Does anyone know of a good gist/blog example reference with some real world examples? I've been good with the documentation except for facets, even though the samples work well.

 I'll give you an example below. Every example has the facet grouping explicitly set:

"facets": {
"colors": { <- How is this grouping field dynamic to just match the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of facets. Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

dadoonet
I'm afraid I did not understand your question.
Here is some comment.
"facets": {
"colors": { <- this is a name you give to the facet
"terms": {
"field": "colors", <- the field you compute on
"all_terms": true
}
}
What is your concern here?

You just add this facet request with the search request and you get back hits and computed facet (with the same name you give).


HTH 
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 6 févr. 2013 à 22:00, [hidden email] a écrit :

It might be because I'm low on energy, but I'm having a real tough time reconciling facet structure in my head right now. Does anyone know of a good gist/blog example reference with some real world examples? I've been good with the documentation except for facets, even though the samples work well.

 I'll give you an example below. Every example has the facet grouping explicitly set:

"facets": {
"colors": { <- How is this grouping field dynamic to just match the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of facets. Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Justin Treher
In the end, I'm trying to get the faceting count  to work "automatically" when I'm not filtering by facet, but just using an ad hoc query.

Every one of my documents has a child object called "facets" that has all the key:value attributes that belong to it to describe it. One document might have the colors key and another might have the genre key. With an ad hoc query, I'd like ES to dive into the facets child object and group together any matches just like it does with the standard syntax. However, the facet "group" name will be one of the thousands of "keys."

input:
title: Mayhem
facets:
 genre:horror
 format: vhs
title: VHS player
facets:
 remote control: yes
 format: vhs

Search: Mayhem OR Player

output:
....
facets
  genre
    horror(1)
  format
    vhs(2)
  remote control
    yes(1)


On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
I'm afraid I did not understand your question.
Here is some comment.
"facets": {
"colors": { <- this is a name you give to the facet
"terms": {
"field": "colors", <- the field you compute on
"all_terms": true
}
}
What is your concern here?

You just add this facet request with the search request and you get back hits and computed facet (with the same name you give).


HTH 
--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 6 févr. 2013 à 22:00, <a href="javascript:" target="_blank" gdf-obfuscated-mailto="ARsIumvOsmQJ">jtr...@... a écrit :

It might be because I'm low on energy, but I'm having a real tough time reconciling facet structure in my head right now. Does anyone know of a good gist/blog example reference with some real world examples? I've been good with the documentation except for facets, even though the samples work well.

 I'll give you an example below. Every example has the facet grouping explicitly set:

"facets": {
"colors": { <- How is this grouping field dynamic to just match the field name?
"terms": {
"field": "colors",
"all_terms": true
}
}

I'm also struggling with documents that can contain any variety of facets. Some have none, some have many.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="ARsIumvOsmQJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Matt Weber-2
So you don't know what fields you want to facet on?  It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there?  If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:

> In the end, I'm trying to get the faceting count  to work "automatically"
> when I'm not filtering by facet, but just using an ad hoc query.
>
> Every one of my documents has a child object called "facets" that has all
> the key:value attributes that belong to it to describe it. One document
> might have the colors key and another might have the genre key. With an ad
> hoc query, I'd like ES to dive into the facets child object and group
> together any matches just like it does with the standard syntax. However,
> the facet "group" name will be one of the thousands of "keys."
>
> input:
> title: Mayhem
> facets:
>  genre:horror
>  format: vhs
> title: VHS player
> facets:
>  remote control: yes
>  format: vhs
>
> Search: Mayhem OR Player
>
> output:
> ....
> facets
>   genre
>     horror(1)
>   format
>     vhs(2)
>   remote control
>     yes(1)
>
>
>
> On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>
>> I'm afraid I did not understand your question.
>> Here is some comment.
>>
>>     "facets": {
>>         "colors": {  <- this is a name you give to the facet
>>             "terms": {
>>                 "field": "colors", <- the field you compute on
>>                 "all_terms": true
>>             }
>>         }
>>
>> What is your concern here?
>>
>> You just add this facet request with the search request and you get back
>> hits and computed facet (with the same name you give).
>>
>> Did you read this page?
>> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>
>> HTH
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>
>> It might be because I'm low on energy, but I'm having a real tough time
>> reconciling facet structure in my head right now. Does anyone know of a good
>> gist/blog example reference with some real world examples? I've been good
>> with the documentation except for facets, even though the samples work well.
>>
>>  I'll give you an example below. Every example has the facet grouping
>> explicitly set:
>>
>>     "facets": {
>>         "colors": {  <- How is this grouping field dynamic to just match
>> the field name?
>>             "terms": {
>>                 "field": "colors",
>>                 "all_terms": true
>>             }
>>         }
>>
>> I'm also struggling with documents that can contain any variety of facets.
>> Some have none, some have many.
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [hidden email].
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Justin Treher
When I'm actually doing a facet filter, I will know which facet(s) to filter on, but upon the initial ad hoc query, I will need to present the facets to the user, grouped by count. Regardless, even when using a facet filter or two, I'd still like to present the remaining, unused facets to the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
So you don't know what fields you want to facet on?  It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there?  If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM,  <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="1JQpqAHBFcYJ">jtr...@...> wrote:

> In the end, I'm trying to get the faceting count  to work "automatically"
> when I'm not filtering by facet, but just using an ad hoc query.
>
> Every one of my documents has a child object called "facets" that has all
> the key:value attributes that belong to it to describe it. One document
> might have the colors key and another might have the genre key. With an ad
> hoc query, I'd like ES to dive into the facets child object and group
> together any matches just like it does with the standard syntax. However,
> the facet "group" name will be one of the thousands of "keys."
>
> input:
> title: Mayhem
> facets:
>  genre:horror
>  format: vhs
> title: VHS player
> facets:
>  remote control: yes
>  format: vhs
>
> Search: Mayhem OR Player
>
> output:
> ....
> facets
>   genre
>     horror(1)
>   format
>     vhs(2)
>   remote control
>     yes(1)
>
>
>
> On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>
>> I'm afraid I did not understand your question.
>> Here is some comment.
>>
>>     "facets": {
>>         "colors": {  <- this is a name you give to the facet
>>             "terms": {
>>                 "field": "colors", <- the field you compute on
>>                 "all_terms": true
>>             }
>>         }
>>
>> What is your concern here?
>>
>> You just add this facet request with the search request and you get back
>> hits and computed facet (with the same name you give).
>>
>> Did you read this page?
>> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>
>> HTH
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>
>> It might be because I'm low on energy, but I'm having a real tough time
>> reconciling facet structure in my head right now. Does anyone know of a good
>> gist/blog example reference with some real world examples? I've been good
>> with the documentation except for facets, even though the samples work well.
>>
>>  I'll give you an example below. Every example has the facet grouping
>> explicitly set:
>>
>>     "facets": {
>>         "colors": {  <- How is this grouping field dynamic to just match
>> the field name?
>>             "terms": {
>>                 "field": "colors",
>>                 "all_terms": true
>>             }
>>         }
>>
>> I'm also struggling with documents that can contain any variety of facets.
>> Some have none, some have many.
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="1JQpqAHBFcYJ">elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Justin Treher
@Matt Weber

Were you suggesting that I run a query to get ids and then run a subsequent query using facets found in those results to get the appropriate count, restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, [hidden email] wrote:
When I'm actually doing a facet filter, I will know which facet(s) to filter on, but upon the initial ad hoc query, I will need to present the facets to the user, grouped by count. Regardless, even when using a facet filter or two, I'd still like to present the remaining, unused facets to the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
So you don't know what fields you want to facet on?  It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there?  If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:

> In the end, I'm trying to get the faceting count  to work "automatically"
> when I'm not filtering by facet, but just using an ad hoc query.
>
> Every one of my documents has a child object called "facets" that has all
> the key:value attributes that belong to it to describe it. One document
> might have the colors key and another might have the genre key. With an ad
> hoc query, I'd like ES to dive into the facets child object and group
> together any matches just like it does with the standard syntax. However,
> the facet "group" name will be one of the thousands of "keys."
>
> input:
> title: Mayhem
> facets:
>  genre:horror
>  format: vhs
> title: VHS player
> facets:
>  remote control: yes
>  format: vhs
>
> Search: Mayhem OR Player
>
> output:
> ....
> facets
>   genre
>     horror(1)
>   format
>     vhs(2)
>   remote control
>     yes(1)
>
>
>
> On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>
>> I'm afraid I did not understand your question.
>> Here is some comment.
>>
>>     "facets": {
>>         "colors": {  <- this is a name you give to the facet
>>             "terms": {
>>                 "field": "colors", <- the field you compute on
>>                 "all_terms": true
>>             }
>>         }
>>
>> What is your concern here?
>>
>> You just add this facet request with the search request and you get back
>> hits and computed facet (with the same name you give).
>>
>> Did you read this page?
>> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>
>> HTH
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>
>> It might be because I'm low on energy, but I'm having a real tough time
>> reconciling facet structure in my head right now. Does anyone know of a good
>> gist/blog example reference with some real world examples? I've been good
>> with the documentation except for facets, even though the samples work well.
>>
>>  I'll give you an example below. Every example has the facet grouping
>> explicitly set:
>>
>>     "facets": {
>>         "colors": {  <- How is this grouping field dynamic to just match
>> the field name?
>>             "terms": {
>>                 "field": "colors",
>>                 "all_terms": true
>>             }
>>         }
>>
>> I'm also struggling with documents that can contain any variety of facets.
>> Some have none, some have many.
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Justin Treher
I did find this thread with a reply from the master himself. So, there is no out of the box way to do what all of us what for an "ecommerce" style mixed bag lot of filters for thousands of products.
https://groups.google.com/d/topic/elasticsearch/3_Oer8hT-9U/discussion

I agree with the OP about the ugly concatenation, but I also see this as a suggested SOLR solution on StackOverflow. I think it would be nice to post the "work around" within the documentation because it's really not clear that it's not possible. Another to do for myself when I look at editing the docs!



On Tuesday, February 12, 2013 1:02:26 PM UTC-5, [hidden email] wrote:
@Matt Weber

Were you suggesting that I run a query to get ids and then run a subsequent query using facets found in those results to get the appropriate count, restricting the scope of the count to the id set?

Thanks!

On Thursday, February 7, 2013 9:39:17 AM UTC-5, [hidden email] wrote:
When I'm actually doing a facet filter, I will know which facet(s) to filter on, but upon the initial ad hoc query, I will need to present the facets to the user, grouped by count. Regardless, even when using a facet filter or two, I'd still like to present the remaining, unused facets to the user grouped by count.

On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
So you don't know what fields you want to facet on?  It sounds like
you want to facet on any field that is in the "facets" object, but you
don't really know what is there?  If this is fact what you are trying
to do, then you could always pull the mapping for your index and parse
out any dynamic fields under the facets object then use those to
create the facets in your query.

On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:

> In the end, I'm trying to get the faceting count  to work "automatically"
> when I'm not filtering by facet, but just using an ad hoc query.
>
> Every one of my documents has a child object called "facets" that has all
> the key:value attributes that belong to it to describe it. One document
> might have the colors key and another might have the genre key. With an ad
> hoc query, I'd like ES to dive into the facets child object and group
> together any matches just like it does with the standard syntax. However,
> the facet "group" name will be one of the thousands of "keys."
>
> input:
> title: Mayhem
> facets:
>  genre:horror
>  format: vhs
> title: VHS player
> facets:
>  remote control: yes
>  format: vhs
>
> Search: Mayhem OR Player
>
> output:
> ....
> facets
>   genre
>     horror(1)
>   format
>     vhs(2)
>   remote control
>     yes(1)
>
>
>
> On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>
>> I'm afraid I did not understand your question.
>> Here is some comment.
>>
>>     "facets": {
>>         "colors": {  <- this is a name you give to the facet
>>             "terms": {
>>                 "field": "colors", <- the field you compute on
>>                 "all_terms": true
>>             }
>>         }
>>
>> What is your concern here?
>>
>> You just add this facet request with the search request and you get back
>> hits and computed facet (with the same name you give).
>>
>> Did you read this page?
>> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>
>> HTH
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>
>> It might be because I'm low on energy, but I'm having a real tough time
>> reconciling facet structure in my head right now. Does anyone know of a good
>> gist/blog example reference with some real world examples? I've been good
>> with the documentation except for facets, even though the samples work well.
>>
>>  I'll give you an example below. Every example has the facet grouping
>> explicitly set:
>>
>>     "facets": {
>>         "colors": {  <- How is this grouping field dynamic to just match
>> the field name?
>>             "terms": {
>>                 "field": "colors",
>>                 "all_terms": true
>>             }
>>         }
>>
>> I'm also struggling with documents that can contain any variety of facets.
>> Some have none, some have many.
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Matt Weber-2
In reply to this post by Justin Treher
I still am not 100% clear on what you are trying to accomplish.  It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set.  Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names.  I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object.  You can then use these names at
query time to setup your facets.  Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM,  <[hidden email]> wrote:

> @Matt Weber
>
> Were you suggesting that I run a query to get ids and then run a subsequent
> query using facets found in those results to get the appropriate count,
> restricting the scope of the count to the id set?
>
> Thanks!
>
>
> On Thursday, February 7, 2013 9:39:17 AM UTC-5, [hidden email] wrote:
>>
>> When I'm actually doing a facet filter, I will know which facet(s) to
>> filter on, but upon the initial ad hoc query, I will need to present the
>> facets to the user, grouped by count. Regardless, even when using a facet
>> filter or two, I'd still like to present the remaining, unused facets to the
>> user grouped by count.
>>
>> On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
>>>
>>> So you don't know what fields you want to facet on?  It sounds like
>>> you want to facet on any field that is in the "facets" object, but you
>>> don't really know what is there?  If this is fact what you are trying
>>> to do, then you could always pull the mapping for your index and parse
>>> out any dynamic fields under the facets object then use those to
>>> create the facets in your query.
>>>
>>> On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:
>>> > In the end, I'm trying to get the faceting count  to work
>>> > "automatically"
>>> > when I'm not filtering by facet, but just using an ad hoc query.
>>> >
>>> > Every one of my documents has a child object called "facets" that has
>>> > all
>>> > the key:value attributes that belong to it to describe it. One document
>>> > might have the colors key and another might have the genre key. With an
>>> > ad
>>> > hoc query, I'd like ES to dive into the facets child object and group
>>> > together any matches just like it does with the standard syntax.
>>> > However,
>>> > the facet "group" name will be one of the thousands of "keys."
>>> >
>>> > input:
>>> > title: Mayhem
>>> > facets:
>>> >  genre:horror
>>> >  format: vhs
>>> > title: VHS player
>>> > facets:
>>> >  remote control: yes
>>> >  format: vhs
>>> >
>>> > Search: Mayhem OR Player
>>> >
>>> > output:
>>> > ....
>>> > facets
>>> >   genre
>>> >     horror(1)
>>> >   format
>>> >     vhs(2)
>>> >   remote control
>>> >     yes(1)
>>> >
>>> >
>>> >
>>> > On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>> >>
>>> >> I'm afraid I did not understand your question.
>>> >> Here is some comment.
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- this is a name you give to the facet
>>> >>             "terms": {
>>> >>                 "field": "colors", <- the field you compute on
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> What is your concern here?
>>> >>
>>> >> You just add this facet request with the search request and you get
>>> >> back
>>> >> hits and computed facet (with the same name you give).
>>> >>
>>> >> Did you read this page?
>>> >> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>> >>
>>> >> HTH
>>> >> --
>>> >> David ;-)
>>> >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>> >>
>>> >>
>>> >> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>> >>
>>> >> It might be because I'm low on energy, but I'm having a real tough
>>> >> time
>>> >> reconciling facet structure in my head right now. Does anyone know of
>>> >> a good
>>> >> gist/blog example reference with some real world examples? I've been
>>> >> good
>>> >> with the documentation except for facets, even though the samples work
>>> >> well.
>>> >>
>>> >>  I'll give you an example below. Every example has the facet grouping
>>> >> explicitly set:
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- How is this grouping field dynamic to just
>>> >> match
>>> >> the field name?
>>> >>             "terms": {
>>> >>                 "field": "colors",
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> I'm also struggling with documents that can contain any variety of
>>> >> facets.
>>> >> Some have none, some have many.
>>> >>
>>> >> Thanks!
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "elasticsearch" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to [hidden email].
>>> >>
>>> >> For more options, visit https://groups.google.com/groups/opt_out.
>>> >>
>>> >>
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "elasticsearch" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> > an
>>> > email to [hidden email].
>>> > For more options, visit https://groups.google.com/groups/opt_out.
>>> >
>>> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Justin Treher
@Matt,

I might have a grip on it now. I think the major disjunction that I and others in my situation are having is the fundamental concept of faceted navigation.

There are some sites, such as a real estate site or a travel site which use the same standard list of facets for every search result. location, price, to, from, # of bedrooms, "other options" etc, etc. The facet toolkit out-of-the-box provided by ES works for this type of navigation. That's what the documentation uses for an example.

However, for organizations that do ecommerce, your typical faceted navigation is dependent on the category or products that you are browsing. And, for an "ad hoc" query, you might pull in products from multiple domains, therefore the facets are dynamic based on the results. For instance, if you sell speakers, shirts, and ovens, you have facets for those domains. There is no out of the box way to tell SOLR or ES as far as I can tell, to use complex facets consisting of key/value pairs (except for the mathematical ones). For instance, if I could give ES the ideal structure below and have it consume a key/value field for facet name/type it would work out. The workaround to give ES a concatenation of key+value works, it's just that it feels wrong. So, I store both the ideal version, when I know the facet and the workaround version for ad hoc queries to get counts. The application can break apart the workaround easily enough.

Ideal:
{facets:[
  {facet_name:name1,
   facet_value:value1},
  {facet_name:name2,
  facet_value:value2}
]

Workaround to "flatten facets". In this case ### is just a place holder for the app to break apart the strings.
facets:[name1###value1,name2###value2]

Finally, I  know that sites like Amazon force you to pick a category before presenting domain facets. This would give one the ability to find facets for a category and setup a proper facet query for ES. However, our current specs don't allow for this, but it's certainly something to consider.

On Tuesday, February 12, 2013 2:50:14 PM UTC-5, Matt Weber wrote:
I still am not 100% clear on what you are trying to accomplish.  It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set.  Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names.  I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object.  You can then use these names at
query time to setup your facets.  Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM,  <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="R5svCVLpAVgJ">jtr...@...> wrote:

> @Matt Weber
>
> Were you suggesting that I run a query to get ids and then run a subsequent
> query using facets found in those results to get the appropriate count,
> restricting the scope of the count to the id set?
>
> Thanks!
>
>
> On Thursday, February 7, 2013 9:39:17 AM UTC-5, [hidden email] wrote:
>>
>> When I'm actually doing a facet filter, I will know which facet(s) to
>> filter on, but upon the initial ad hoc query, I will need to present the
>> facets to the user, grouped by count. Regardless, even when using a facet
>> filter or two, I'd still like to present the remaining, unused facets to the
>> user grouped by count.
>>
>> On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
>>>
>>> So you don't know what fields you want to facet on?  It sounds like
>>> you want to facet on any field that is in the "facets" object, but you
>>> don't really know what is there?  If this is fact what you are trying
>>> to do, then you could always pull the mapping for your index and parse
>>> out any dynamic fields under the facets object then use those to
>>> create the facets in your query.
>>>
>>> On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:
>>> > In the end, I'm trying to get the faceting count  to work
>>> > "automatically"
>>> > when I'm not filtering by facet, but just using an ad hoc query.
>>> >
>>> > Every one of my documents has a child object called "facets" that has
>>> > all
>>> > the key:value attributes that belong to it to describe it. One document
>>> > might have the colors key and another might have the genre key. With an
>>> > ad
>>> > hoc query, I'd like ES to dive into the facets child object and group
>>> > together any matches just like it does with the standard syntax.
>>> > However,
>>> > the facet "group" name will be one of the thousands of "keys."
>>> >
>>> > input:
>>> > title: Mayhem
>>> > facets:
>>> >  genre:horror
>>> >  format: vhs
>>> > title: VHS player
>>> > facets:
>>> >  remote control: yes
>>> >  format: vhs
>>> >
>>> > Search: Mayhem OR Player
>>> >
>>> > output:
>>> > ....
>>> > facets
>>> >   genre
>>> >     horror(1)
>>> >   format
>>> >     vhs(2)
>>> >   remote control
>>> >     yes(1)
>>> >
>>> >
>>> >
>>> > On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>> >>
>>> >> I'm afraid I did not understand your question.
>>> >> Here is some comment.
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- this is a name you give to the facet
>>> >>             "terms": {
>>> >>                 "field": "colors", <- the field you compute on
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> What is your concern here?
>>> >>
>>> >> You just add this facet request with the search request and you get
>>> >> back
>>> >> hits and computed facet (with the same name you give).
>>> >>
>>> >> Did you read this page?
>>> >> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>> >>
>>> >> HTH
>>> >> --
>>> >> David ;-)
>>> >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>> >>
>>> >>
>>> >> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>> >>
>>> >> It might be because I'm low on energy, but I'm having a real tough
>>> >> time
>>> >> reconciling facet structure in my head right now. Does anyone know of
>>> >> a good
>>> >> gist/blog example reference with some real world examples? I've been
>>> >> good
>>> >> with the documentation except for facets, even though the samples work
>>> >> well.
>>> >>
>>> >>  I'll give you an example below. Every example has the facet grouping
>>> >> explicitly set:
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- How is this grouping field dynamic to just
>>> >> match
>>> >> the field name?
>>> >>             "terms": {
>>> >>                 "field": "colors",
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> I'm also struggling with documents that can contain any variety of
>>> >> facets.
>>> >> Some have none, some have many.
>>> >>
>>> >> Thanks!
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "elasticsearch" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to elasticsearc...@googlegroups.com.
>>> >>
>>> >> For more options, visit https://groups.google.com/groups/opt_out.
>>> >>
>>> >>
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "elasticsearch" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> > an
>>> > email to elasticsearc...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/groups/opt_out.
>>> >
>>> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="R5svCVLpAVgJ">elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

Ivan Brusic
We do something similar to your workaround for our ecommerce app. All our dynamic product attributes are faceted against one field. Our app then flattens things on the client side before returning the data to the user. For efficiency, we do not use strings, but bit-shifted numerical values. A client side mapping converts numeric ids into text values since our taxonomy is known.

-- 
Ivan


On Wed, Feb 13, 2013 at 5:19 AM, <[hidden email]> wrote:
@Matt,

I might have a grip on it now. I think the major disjunction that I and others in my situation are having is the fundamental concept of faceted navigation.

There are some sites, such as a real estate site or a travel site which use the same standard list of facets for every search result. location, price, to, from, # of bedrooms, "other options" etc, etc. The facet toolkit out-of-the-box provided by ES works for this type of navigation. That's what the documentation uses for an example.

However, for organizations that do ecommerce, your typical faceted navigation is dependent on the category or products that you are browsing. And, for an "ad hoc" query, you might pull in products from multiple domains, therefore the facets are dynamic based on the results. For instance, if you sell speakers, shirts, and ovens, you have facets for those domains. There is no out of the box way to tell SOLR or ES as far as I can tell, to use complex facets consisting of key/value pairs (except for the mathematical ones). For instance, if I could give ES the ideal structure below and have it consume a key/value field for facet name/type it would work out. The workaround to give ES a concatenation of key+value works, it's just that it feels wrong. So, I store both the ideal version, when I know the facet and the workaround version for ad hoc queries to get counts. The application can break apart the workaround easily enough.

Ideal:
{facets:[
  {facet_name:name1,
   facet_value:value1},
  {facet_name:name2,
  facet_value:value2}
]

Workaround to "flatten facets". In this case ### is just a place holder for the app to break apart the strings.
facets:[name1###value1,name2###value2]

Finally, I  know that sites like Amazon force you to pick a category before presenting domain facets. This would give one the ability to find facets for a category and setup a proper facet query for ES. However, our current specs don't allow for this, but it's certainly something to consider.

On Tuesday, February 12, 2013 2:50:14 PM UTC-5, Matt Weber wrote:
I still am not 100% clear on what you are trying to accomplish.  It
sounded like you have a field called "facets" that is an object.
Within that there can be any number of fields (genre, format,
has_remote, etc), but you don't necessarily know these field names
because each product you index can have it's own set.  Since ES
requires that you know what field you want to facet on at query time,
ie. you need to specify a field name, you are out of luck because you
don't know the field names.  I was suggesting you pull the mapping of
the index you are searching across, then inspect the names of the
fields within the "facets" object.  You can then use these names at
query time to setup your facets.  Now I wouldn't pull the mapping for
every query, but you could do it on some pre-defined interval and
cache the results within your application.

On Tue, Feb 12, 2013 at 10:02 AM,  <[hidden email]> wrote:

> @Matt Weber
>
> Were you suggesting that I run a query to get ids and then run a subsequent
> query using facets found in those results to get the appropriate count,
> restricting the scope of the count to the id set?
>
> Thanks!
>
>
> On Thursday, February 7, 2013 9:39:17 AM UTC-5, [hidden email] wrote:
>>
>> When I'm actually doing a facet filter, I will know which facet(s) to
>> filter on, but upon the initial ad hoc query, I will need to present the
>> facets to the user, grouped by count. Regardless, even when using a facet
>> filter or two, I'd still like to present the remaining, unused facets to the
>> user grouped by count.
>>
>> On Wednesday, February 6, 2013 5:31:33 PM UTC-5, Matt Weber wrote:
>>>
>>> So you don't know what fields you want to facet on?  It sounds like
>>> you want to facet on any field that is in the "facets" object, but you
>>> don't really know what is there?  If this is fact what you are trying
>>> to do, then you could always pull the mapping for your index and parse
>>> out any dynamic fields under the facets object then use those to
>>> create the facets in your query.
>>>
>>> On Wed, Feb 6, 2013 at 1:34 PM,  <[hidden email]> wrote:
>>> > In the end, I'm trying to get the faceting count  to work
>>> > "automatically"
>>> > when I'm not filtering by facet, but just using an ad hoc query.
>>> >
>>> > Every one of my documents has a child object called "facets" that has
>>> > all
>>> > the key:value attributes that belong to it to describe it. One document
>>> > might have the colors key and another might have the genre key. With an
>>> > ad
>>> > hoc query, I'd like ES to dive into the facets child object and group
>>> > together any matches just like it does with the standard syntax.
>>> > However,
>>> > the facet "group" name will be one of the thousands of "keys."
>>> >
>>> > input:
>>> > title: Mayhem
>>> > facets:
>>> >  genre:horror
>>> >  format: vhs
>>> > title: VHS player
>>> > facets:
>>> >  remote control: yes
>>> >  format: vhs
>>> >
>>> > Search: Mayhem OR Player
>>> >
>>> > output:
>>> > ....
>>> > facets
>>> >   genre
>>> >     horror(1)
>>> >   format
>>> >     vhs(2)
>>> >   remote control
>>> >     yes(1)
>>> >
>>> >
>>> >
>>> > On Wednesday, February 6, 2013 4:12:51 PM UTC-5, David Pilato wrote:
>>> >>
>>> >> I'm afraid I did not understand your question.
>>> >> Here is some comment.
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- this is a name you give to the facet
>>> >>             "terms": {
>>> >>                 "field": "colors", <- the field you compute on
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> What is your concern here?
>>> >>
>>> >> You just add this facet request with the search request and you get
>>> >> back
>>> >> hits and computed facet (with the same name you give).
>>> >>
>>> >> Did you read this page?
>>> >> http://www.elasticsearch.org/guide/reference/api/search/facets/
>>> >>
>>> >> HTH
>>> >> --
>>> >> David ;-)
>>> >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>> >>
>>> >>
>>> >> Le 6 févr. 2013 à 22:00, [hidden email] a écrit :
>>> >>
>>> >> It might be because I'm low on energy, but I'm having a real tough
>>> >> time
>>> >> reconciling facet structure in my head right now. Does anyone know of
>>> >> a good
>>> >> gist/blog example reference with some real world examples? I've been
>>> >> good
>>> >> with the documentation except for facets, even though the samples work
>>> >> well.
>>> >>
>>> >>  I'll give you an example below. Every example has the facet grouping
>>> >> explicitly set:
>>> >>
>>> >>     "facets": {
>>> >>         "colors": {  <- How is this grouping field dynamic to just
>>> >> match
>>> >> the field name?
>>> >>             "terms": {
>>> >>                 "field": "colors",
>>> >>                 "all_terms": true
>>> >>             }
>>> >>         }
>>> >>
>>> >> I'm also struggling with documents that can contain any variety of
>>> >> facets.
>>> >> Some have none, some have many.
>>> >>
>>> >> Thanks!
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> >> Groups
>>> >> "elasticsearch" group.
>>> >> To unsubscribe from this group and stop receiving emails from it, send
>>> >> an
>>> >> email to elasticsearc...@googlegroups.com.
>>> >>
>>> >> For more options, visit https://groups.google.com/groups/opt_out.
>>> >>
>>> >>
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "elasticsearch" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> > an
>>> > email to elasticsearc...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/groups/opt_out.
>>> >
>>> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Facets have got me down

my3sons
In reply to this post by Justin Treher
I apologize in advance if I should have started new thread with this question, but it does have some correlation to the above.  I also apologize for the long winded post, I just want to make sure I get the message across

Just a quick background on our solution here. We have built a Spring web app that sits on top of an ElasticSearch cluster that will allow the Search Business team to define all the facets, synonyms, query boosts, recommended products, etc,  for the Search solution at their respective ecommerce site.  For those of you that are familiar with FAST search, our Spring app is similar to their Business Manager Tool. So for facets, using the ES mappings API, we present to the user a list of all indices in the ES cluster, then based on their selection, we then present all the attributes/properties that are included in the selected indices, and then they ultimately pick what attribute/property to use for that particular facet. They then can do all the related stuff (i.e. if range, designate ranges, if term, how many values to initially display, etc).  So we don’t necessarily have the “dynamic facet” issue that has been discussed here, but the challenge we are facing that is somewhat relevant to this discussion is how can we manage what “context” the facet should appear in. What I mean by that is, for a given facet, I only want it to be included if the customer is in the “Electronics” category, or I may want to slot it globally and return it with all queries. The problem is that over time, the business will potentially create hundreds of facets, and we cannot have every request to ES tagged with hundreds of facet filters. Even if performance was not an issue (and in most of our cases, performance is the only issue ), this will not work given the following scenario (and there are others like it).

The search business team will set up multiple price range facets, each intended to only be evaluated and returned given the context the customer may be in. For example, if the customer is in the TVs category, the price range facet may look like:

$0 - $200
$200 - $400
$400 - $600…

But if they are in the cables category, it might look like:
$0 - $5
$5 - $10
$10 - $15…

So the business will create two different facets with the corresponding ranges and then add the appropriate “category” context to it, and therein lies our challenge, how to implement this?

As jtreher mentioned above, if we only needed to support this within the customer’s browse experience (like Amazon), the solution would be rather straight forward, since we would always know what category context the customer is in at any point in time, and we could then extract those facets accordingly from our cache and include in our ES Query.  However, the business requirement is that we also support this model when the customer issues a global/ad-hoc query. In this scenario, we are in a bit of a chicken/egg situation, in that we won’t know what contexts are included until we evaluate the initial results, and only then can we apply the appropriate facet filtering.

Again, for those of you that have worked with FAST and its ecommerce layer, you know that they support all of this out-of-the-box. I don’t know all the nitty, gritty details of how they have implemented their contextual capability, but what I do know and don’t like, is that they require a full re-index when new facets are created or their context is changed. So obviously they are storing some metadata with each document that indicates what facet/context it belongs to, and this is used at query time to derive facets. Therefore, search query performance is optimized, but at a pretty heavy cost with the full re-indexing.

That all being said, I am struggling to find the best solution with ES. The different options I am looking at now are:

1. Two-step query – Invoke global query with category facet to get all potential category contexts, then go to business managed facet cache and find all facets that match categories returned from initial global query and resubmit query again with all the appropriate facet filters.  Pros – should work, Cons – very concerned with performance impact of this with both additional query load and latency.

2. Do something like what FAST does, and that is when business user creates facet with context in business tool, add some metadata to the corresponding documents in index and then leverage that metadata at query time to create necessary facets. This might be similar to the facet flattening solution described above. Pros- query performance should be better than option1, Cons – not clear on what would need to be implemented and if this would work with ES, don’t want to get into situation where re-indexing is required for every facet transaction.

3. Draw line in the sand and only support contextual faceting under “Browse” scenario.

4. Other solutions???

Thanks for any feedback and a big thanks to all that have contributed to ElasticSearch, it totally rocks!
Loading...