[0.90.x] Sorting stress test and no response

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[0.90.x] Sorting stress test and no response

Marek Skorek
Hi, 

I am testing the limitation for field data cache and I have met some problem.

The case is to execute 3 queries at the same time (nearly) with sorting as below:
query1: sorting by 5 fields (3 of them are unique)
query2: sorting by 1 field (1 unique of the 5 from query1)
query3: sorting by 2 fields (2 non-unique of the 5 from query2; just en Enums/Strings - not too many distinct values)

The index size is about 115M docs that matches above queries.
My cluster consists of 3 nodes with 1,5G of Heap (for test purposes only) and the field data cache size is set to 50% of node's heap.

After a while the "OOME: Java heap space" occurs on the node I request with curl.

Despite of above the cluster is still runing and responsive to queries but these 3 queries hangs up (no reponse sice 5 hours) and GC is going crazy on the node requested by these queries.

Is it normal behaviour or maybe I have missed something in my configuration?
index.fielddata.cache: node
indices.fielddata.cache.size: 50%

Attached Bigdesk screenshot from the node1. 
Other nodes seems to be ok (no OOME reported).


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

ES-bigdesk-after OOME.png (115K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [0.90.x] Sorting stress test and no response

Marek Skorek
I have left my computer working for a night and the queries has got still no responses.

After killing curls commands the CPU usage is still about 100% (of 800%). Heap usage is still growing and GC is working all the time.

Addidional information about JVM:
java version "1.7.0_03"
OpenJDK Runtime Environment (IcedTea7 2.1.7) (7u3-2.1.7-1)
OpenJDK 64-Bit Server VM (build 22.0-b10, mixed mode)

W dniu poniedziałek, 3 czerwca 2013 14:54:14 UTC+2 użytkownik Marek Skorek napisał:
Hi, 

I am testing the limitation for field data cache and I have met some problem.

The case is to execute 3 queries at the same time (nearly) with sorting as below:
query1: sorting by 5 fields (3 of them are unique)
query2: sorting by 1 field (1 unique of the 5 from query1)
query3: sorting by 2 fields (2 non-unique of the 5 from query2; just en Enums/Strings - not too many distinct values)

The index size is about 115M docs that matches above queries.
My cluster consists of 3 nodes with 1,5G of Heap (for test purposes only) and the field data cache size is set to 50% of node's heap.

After a while the "OOME: Java heap space" occurs on the node I request with curl.

Despite of above the cluster is still runing and responsive to queries but these 3 queries hangs up (no reponse sice 5 hours) and GC is going crazy on the node requested by these queries.

Is it normal behaviour or maybe I have missed something in my configuration?
index.fielddata.cache: node
indices.fielddata.cache.size: 50%

Attached Bigdesk screenshot from the node1. 
Other nodes seems to be ok (no OOME reported).


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.