Choking of Elasticsearch.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Choking of Elasticsearch.

narinder.izap
Hi there,
     
         We are using Elasticsearch 0.90.6 on our production server, with one replica. We need to import the data via scripts and we have million of records to be saved. We are using the Bulk mode to save the documents to Elasticsearch index. 

Our architecture is using two Databases, One is Mysql, and other is Elasticsearch. The data is first saved in Mysql, then it is sent to Elasticseach for documents savings. But while data import script is running, the data is being saved in Mysql, but not being pushed to Elasticsearch, On investigating the nodes via Elastic HQ, I found the following factors


         The filter eviction is in red colour, saying it has crossed the limit.I need to understand, the two terms used for this filter eviction(filter_cache.eviction and query_total), How are they being calculated and  How can i control them,How it is affecting the data saving and data searching too, or anything else that can cause the document skipping from saving.The search is also very very slow, although,we have given 30 GB of RAM to Elasticsearch server.

Thanks in advance.
Narinder Kaur

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e3aff1cf-d3ff-4037-ac3b-c846a0d00798%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Choking of Elasticsearch.

Binh Ly-2
This metric/warning should have no effect in indexing. The query_total is the number of shard _searches executed so far, and filter cache evictions counts the number of times something in the filter cache is removed/replaced because you have reached the limits - this is affected by filters in your queries. Filter cache evictions are normal as part of running your queries, although the filter cache size can be increased (or decreased) as needed. But before you do that, I'd probably find out first why search is slow by analyzing your queries.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/85f3e0aa-6b65-4376-9b33-a20ac287bfc6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.