Source and Store Level Compression

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Source and Store Level Compression

Mike
I'm using Elasticsearch 19.8, and currently I have "_source" : { "compress" : "true" } in my index mapping.  My average document size is < 1 KB, but I have millions of them making my indices gigabytes in size.

The documentation says that both source and store compression is enabled in v.90+
http://www.elasticsearch.org/guide/reference/index-modules/store.html
http://www.elasticsearch.org/guide/reference/mapping/source-field.html

Is it then recommended to set index.store.compress.stored : true for versions .19.5 - .20 that have this option?
If I set store compression to true, does it then make setting source compression unneeded?  Should I set source=false and store=true instead or both to true?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Source and Store Level Compression

Radu Gheorghe-2
Hello Mike,

If you have small documents with 0.19.8, I think you should use store-level compression only. That will compress your _source as well, and I don't see the point of having both turned on.

From 0.90, Elasticsearch uses a more efficient, Lucene-level field compression. For some more details, here's an interesting blog post:
http://blog.jpountz.net/post/33247161884/efficient-compressed-stored-fields-with-lucene

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Tue, Mar 12, 2013 at 7:32 PM, Mike <[hidden email]> wrote:
I'm using Elasticsearch 19.8, and currently I have "_source" : { "compress" : "true" } in my index mapping.  My average document size is < 1 KB, but I have millions of them making my indices gigabytes in size.

The documentation says that both source and store compression is enabled in v.90+
http://www.elasticsearch.org/guide/reference/index-modules/store.html
http://www.elasticsearch.org/guide/reference/mapping/source-field.html

Is it then recommended to set index.store.compress.stored : true for versions .19.5 - .20 that have this option?
If I set store compression to true, does it then make setting source compression unneeded?  Should I set source=false and store=true instead or both to true?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.