How much memory to allocate to heap?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How much memory to allocate to heap?

ryanogle
Lets say I have a big machine (~65GB of ram).  And I have elasticsearch running on a VERY active applcation (1000s of updates/writes per second, 100s of reads per second).  I'm looking at running this as a service as described here: http://www.elasticsearch.org/guide/reference/setup/installation.html.  It looks like by default, elasticsearch.conf sets heap size to be 1024.  Also, there are additional parameters in the elasticsearch.conf file for initmemory and maxmemory.  What is the recommendation for these as well?

Thanks for your help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: How much memory to allocate to heap?

kimchy
Administrator
The recommended setting is about 50% of the machine memory, so if you have ~65gb, I would recommend giving ES 30gb of memory for heap (just set ES_HEAP_SIZE env var, which sets both min and max to the same value, which is recommended). The reason for the 30gb memory setting is that the JVM can (under 31gb approximately) compress 64bit pointers.

Don't worry about the rest of the memory, it is happily used by the operating for the file system cache, which means disk access speeds up. In order to even improve that further, you can set the index.store.type to mmapfs which will use memory mapped files for the index files.

On Feb 2, 2013, at 10:25 PM, ryano <[hidden email]> wrote:

Lets say I have a big machine (~65GB of ram).  And I have elasticsearch running on a VERY active applcation (1000s of updates/writes per second, 100s of reads per second).  I'm looking at running this as a service as described here: http://www.elasticsearch.org/guide/reference/setup/installation.html.  It looks like by default, elasticsearch.conf sets heap size to be 1024.  Also, there are additional parameters in the elasticsearch.conf file for initmemory and maxmemory.  What is the recommendation for these as well?

Thanks for your help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: How much memory to allocate to heap?

ryanogle
Perfect!  Thanks much!

On Saturday, February 2, 2013 2:04:59 PM UTC-8, kimchy wrote:
The recommended setting is about 50% of the machine memory, so if you have ~65gb, I would recommend giving ES 30gb of memory for heap (just set ES_HEAP_SIZE env var, which sets both min and max to the same value, which is recommended). The reason for the 30gb memory setting is that the JVM can (under 31gb approximately) compress 64bit pointers.

Don't worry about the rest of the memory, it is happily used by the operating for the file system cache, which means disk access speeds up. In order to even improve that further, you can set the index.store.type to mmapfs which will use memory mapped files for the index files.

On Feb 2, 2013, at 10:25 PM, ryano <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="RsatzmG26hQJ">ryano...@...> wrote:

Lets say I have a big machine (~65GB of ram).  And I have elasticsearch running on a VERY active applcation (1000s of updates/writes per second, 100s of reads per second).  I'm looking at running this as a service as described here: http://www.elasticsearch.org/guide/reference/setup/installation.html.  It looks like by default, elasticsearch.conf sets heap size to be 1024.  Also, there are additional parameters in the elasticsearch.conf file for initmemory and maxmemory.  What is the recommendation for these as well?

Thanks for your help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="RsatzmG26hQJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.