Sharding by term/facet

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Sharding by term/facet

Josiah Carlson
Hello,

I've been planning on using ES in a search application, and it could grow very quickly, so I've been looking at ways to pre-shard my data.

Now, it turns out that every document that I'm indexing has a list of string facets, and all of my searches will have at least one required facet, so a natural sharding method would be to shard based on these facets (if a document has 7 facets, I would assume that it could be available on 7 shards).

Questions:
Is there a way to automatically shard based on facets?
If so, how? If not, could I write a plugin to do so?
If I do shard on facets, will routing to the correct shards be done automatically, without going to the entire cluster?
If not, is there a way for me to write a plugin to make this happen?

Thank you,
 - Josiah

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Sharding by term/facet

ppearcy
I think the best you can do is with routing(http://www.elasticsearch.org/guide/reference/mapping/routing-field.html), but that is only a single value per doc. 

I would test around how your use case scales. 

Best Regards,
Paul

On Wednesday, February 6, 2013 12:59:42 PM UTC-7, Josiah Carlson wrote:
Hello,

I've been planning on using ES in a search application, and it could grow very quickly, so I've been looking at ways to pre-shard my data.

Now, it turns out that every document that I'm indexing has a list of string facets, and all of my searches will have at least one required facet, so a natural sharding method would be to shard based on these facets (if a document has 7 facets, I would assume that it could be available on 7 shards).

Questions:
Is there a way to automatically shard based on facets?
If so, how? If not, could I write a plugin to do so?
If I do shard on facets, will routing to the correct shards be done automatically, without going to the entire cluster?
If not, is there a way for me to write a plugin to make this happen?

Thank you,
 - Josiah

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.