multi-core support for elasticsearch

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

multi-core support for elasticsearch

Ashish Nigam-3
I have been using 0.19.8 version for last 1+ year. For last two weeks, I have been running into lot of stability issues with the cluster - unassigned shards, CPU > 100% etc. I am planning to migrate to latest ES version soon. But I have to support this version for few more weeks. 

I noticed that ES java process is only using one core, even though ES node has 6 cores.
Did 0.19.8 version process runs only one core? 
If not, how can I enable ES process to use multi-core?

Thanks
Ashish


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: multi-core support for elasticsearch

joergprante@gmail.com
ES is by default using all available CPU cores. If you see Java consuming a single thread and goes up to 100% without going low again, you have an issue.

Which OS are you running and how much resources (CPU, Memory) have you allocated to ES?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: multi-core support for elasticsearch

Alexander Petrovsky
In reply to this post by Ashish Nigam-3
Hi!

I have the same problem on my singe elasticsearch instance. 

# dpkg -l | grep elas
ii  elasticsearch                      1.4.2                         all          Open Source, Distributed, RESTful Search Engine

# uname -a
Linux s2322.serverel.net 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u2 x86_64 GNU/Linux

# top
Tasks:  98 total,   2 running,  96 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.2 us,  5.8 sy,  0.9 ni, 74.0 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   8178012 total,  6844332 used,  1333680 free,   190952 buffers
KiB Swap:  8000508 total,        0 used,  8000508 free,  5040912 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                                                                              
23694 elastics  20   0 23740 3872 1080 R 100.2  0.0 517:24.56 core                                                                                                                                                 
50459 logstash  39  19 1019m 238m  15m S   3.7  3.0   1:36.33 java                                                                                                                                                 

htop show me only one core consumption.

Can you help me?

среда, 23 октября 2013 г., 15:53:47 UTC+1 пользователь Ashish Nigam написал:
I have been using 0.19.8 version for last 1+ year. For last two weeks, I have been running into lot of stability issues with the cluster - unassigned shards, CPU > 100% etc. I am planning to migrate to latest ES version soon. But I have to support this version for few more weeks. 

I noticed that ES java process is only using one core, even though ES node has 6 cores.
Did 0.19.8 version process runs only one core? 
If not, how can I enable ES process to use multi-core?

Thanks
Ashish


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02254b17-69c0-4c09-82c6-48c06919cf78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: multi-core support for elasticsearch

Alexander Petrovsky
In reply to this post by Ashish Nigam-3
Hi!

I have the same problem on my singe elasticsearch instance. 

# dpkg -l | grep elas
ii  elasticsearch                      1.4.2                         all          Open Source, Distributed, RESTful Search Engine

# uname -a
Linux .... 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u2 x86_64 GNU/Linux

# top
Tasks:  98 total,   2 running,  96 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.2 us,  5.8 sy,  0.9 ni, 74.0 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   8178012 total,  6844332 used,  1333680 free,   190952 buffers
KiB Swap:  8000508 total,        0 used,  8000508 free,  5040912 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                                                                              
23694 elastics  20   0 23740 3872 1080 R 100.2  0.0 517:24.56 core                                                                                                                                                 
50459 logstash  39  19 1019m 238m  15m S   3.7  3.0   1:36.33 java                                                                                                                                                 

htop show me only one core consumption.

Can you help me?

среда, 23 октября 2013 г., 15:53:47 UTC+1 пользователь Ashish Nigam написал:
I have been using 0.19.8 version for last 1+ year. For last two weeks, I have been running into lot of stability issues with the cluster - unassigned shards, CPU > 100% etc. I am planning to migrate to latest ES version soon. But I have to support this version for few more weeks. 

I noticed that ES java process is only using one core, even though ES node has 6 cores.
Did 0.19.8 version process runs only one core? 
If not, how can I enable ES process to use multi-core?

Thanks
Ashish


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51a2808e-44bb-4740-9c72-c63eb15c06d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: multi-core support for elasticsearch

joergprante@gmail.com
How do you observe there is "only one core"?

Elastisearch uses many threads by default and as many cores as possible.

Jörg

On Fri, Mar 13, 2015 at 12:40 PM, Alexander Petrovsky <[hidden email]> wrote:
Hi!

I have the same problem on my singe elasticsearch instance. 

# dpkg -l | grep elas
ii  elasticsearch                      1.4.2                         all          Open Source, Distributed, RESTful Search Engine

# uname -a
Linux .... 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u2 x86_64 GNU/Linux

# top
Tasks:  98 total,   2 running,  96 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.2 us,  5.8 sy,  0.9 ni, 74.0 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   8178012 total,  6844332 used,  1333680 free,   190952 buffers
KiB Swap:  8000508 total,        0 used,  8000508 free,  5040912 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                                                                              
23694 elastics  20   0 23740 3872 1080 R 100.2  0.0 517:24.56 core                                                                                                                                                 
50459 logstash  39  19 1019m 238m  15m S   3.7  3.0   1:36.33 java                                                                                                                                                 

htop show me only one core consumption.

Can you help me?

среда, 23 октября 2013 г., 15:53:47 UTC+1 пользователь Ashish Nigam написал:
I have been using 0.19.8 version for last 1+ year. For last two weeks, I have been running into lot of stability issues with the cluster - unassigned shards, CPU > 100% etc. I am planning to migrate to latest ES version soon. But I have to support this version for few more weeks. 

I noticed that ES java process is only using one core, even though ES node has 6 cores.
Did 0.19.8 version process runs only one core? 
If not, how can I enable ES process to use multi-core?

Thanks
Ashish


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51a2808e-44bb-4740-9c72-c63eb15c06d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHW%2BWoe6SMHXyDk%2BZvURk0xCpakOGHBrEPp3_bTHFLLAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: multi-core support for elasticsearch

Aaron Mefford
I saw this kind of behavior with some older versions of Java, but I mean old, not something I would expect to run ES, but then again your on an old build of ES so I thought I would mention it.



--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0d7720ea-9c00-4360-92a2-2a169a2b6497%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.