Zen Discovery settings in EC2 and escaping from Java API

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Zen Discovery settings in EC2 and escaping from Java API


Before asking my questions, I want to describe my situation.

I deployed elasticsearch in the production today but within 3 hours, I had to turn off all the clients because elasticsearch cluster got crashed.

Now, I am using elasticsearch for realtime log analysis. It should handle multi billion rows in a day and I deployed elasitcsearch to more than 30 instances in EC2.
I changed several settings including number of shards and replication but most settings were default.

First, its performance was really good but when the clients started to be throttling, whole elasticsearch cluster was gone to unhealthy. I checked the log and there were lots of fault detection but every instance was alive.

So my plan is to change the zen discovery setting as the following:

transport.tcp.connect_timeout: 10s 
discovery.zen.minimum_master_nodes: 5
discovery.zen.ping.timeout: 60s
discovery.zen.fd.ping_interval: 60s
discovery.zen.fd.ping_timeout: 60s
discovery.zen.ping.multicast.enabled: false

My second question is, when I restart servers, do I have to restart the client too? Even though I restarted servers, the clients couldn't get connected. If I want the clients totally independent from servers, I have to use REST API, what's the best way to generate JSON string request? Is there any method from Java API to JSON string such as ToXContent interface?

Am I on the right track?

Thank you
Best, Jae