Questions about dedicated master & client node

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about dedicated master & client node

Xudong You
Right now we only need 4 ES nodes due to the small data volume, and all 4 nodes are master & data nodes.

Q1:
I am wondering in this case, is it necessary to have dedicated master and client node? Any benefit of having dedicated master node?

Some one said that dedicated master nodes (say, three master nodes) is helpful to avoid the split brain issue, but even we have NO dedicated master nodes, we can also avoid the split brain by setting the discovery.zen.minimum_master_nodes to a appropriate value.

Q2:
Similarly, is dedicated client node really necessary in our 4 nodes case? Any benefit of allocating dedicated client node?

Thanks!

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about dedicated master & client node

Nikolas Everett
Dedicated master nodes are super convenient if you have the it infrastructure to host them on shared machines because they are very low load and its useful to be able to restart the master nodes quickly. We don't have that kind of infrastructure and our cluster is pretty large and not having it has bitten us once or twice but its not a huge problem.



On Fri, May 29, 2015 at 10:44 AM, Xudong You <[hidden email]> wrote:
Right now we only need 4 ES nodes due to the small data volume, and all 4 nodes are master & data nodes.

Q1:
I am wondering in this case, is it necessary to have dedicated master and client node? Any benefit of having dedicated master node?

Some one said that dedicated master nodes (say, three master nodes) is helpful to avoid the split brain issue, but even we have NO dedicated master nodes, we can also avoid the split brain by setting the discovery.zen.minimum_master_nodes to a appropriate value.

Q2:
Similarly, is dedicated client node really necessary in our 4 nodes case? Any benefit of allocating dedicated client node?

Thanks!

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0gvw%2BKvgGzX7Fq32u04ofQy5t9BAvZUr6_PvZRGFm_tA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about dedicated master & client node

Xudong You
Thanks Nikolas,
How do you think about dedicated "client node" (the so called load balance node)? Any benefit of dedicated client node? Seems to me, round robin to data nodes is good enough.

On Friday, May 29, 2015 at 10:55:01 PM UTC+8, Nikolas Everett wrote:
Dedicated master nodes are super convenient if you have the it infrastructure to host them on shared machines because they are very low load and its useful to be able to restart the master nodes quickly. We don't have that kind of infrastructure and our cluster is pretty large and not having it has bitten us once or twice but its not a huge problem.



On Fri, May 29, 2015 at 10:44 AM, Xudong You <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="vfV_hjK0HR4J" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">xudon...@...> wrote:
Right now we only need 4 ES nodes due to the small data volume, and all 4 nodes are master & data nodes.

Q1:
I am wondering in this case, is it necessary to have dedicated master and client node? Any benefit of having dedicated master node?

Some one said that dedicated master nodes (say, three master nodes) is helpful to avoid the split brain issue, but even we have NO dedicated master nodes, we can also avoid the split brain by setting the discovery.zen.minimum_master_nodes to a appropriate value.

Q2:
Similarly, is dedicated client node really necessary in our 4 nodes case? Any benefit of allocating dedicated client node?

Thanks!

--
Please update your bookmarks! We have moved to <a href="https://discuss.elastic.co/" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;">https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="vfV_hjK0HR4J" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;">https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about dedicated master & client node

James Macdonald
If it is good enough for you, it is good enough for you. I will just give you one anecdote: We implemented 3 dedicated clients on a 9 data node cluster and got a 2x performance improvement. Moving the query coordination, network io (has to receive data from every shard), and combination of results (aggs and sorts) off of the nodes providing the results is very helpful. 

James



On Sat, May 30, 2015 at 9:11 AM, Xudong You <[hidden email]> wrote:
Thanks Nikolas,
How do you think about dedicated "client node" (the so called load balance node)? Any benefit of dedicated client node? Seems to me, round robin to data nodes is good enough.

On Friday, May 29, 2015 at 10:55:01 PM UTC+8, Nikolas Everett wrote:
Dedicated master nodes are super convenient if you have the it infrastructure to host them on shared machines because they are very low load and its useful to be able to restart the master nodes quickly. We don't have that kind of infrastructure and our cluster is pretty large and not having it has bitten us once or twice but its not a huge problem.



On Fri, May 29, 2015 at 10:44 AM, Xudong You <[hidden email]> wrote:
Right now we only need 4 ES nodes due to the small data volume, and all 4 nodes are master & data nodes.

Q1:
I am wondering in this case, is it necessary to have dedicated master and client node? Any benefit of having dedicated master node?

Some one said that dedicated master nodes (say, three master nodes) is helpful to avoid the split brain issue, but even we have NO dedicated master nodes, we can also avoid the split brain by setting the discovery.zen.minimum_master_nodes to a appropriate value.

Q2:
Similarly, is dedicated client node really necessary in our 4 nodes case? Any benefit of allocating dedicated client node?

Thanks!

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAABsnTbQispJSH%3D7_wbk-W5%2BmMq1_4Yy2Mxeh8RL%2BAAYeaRx6g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Questions about dedicated master & client node

Xudong You
Thanks James's sharing.
Does your client node has same performance (CPU & memory) as data node or lower perf?

On Sunday, May 31, 2015 at 12:25:58 AM UTC+8, James Macdonald wrote:
If it is good enough for you, it is good enough for you. I will just give you one anecdote: We implemented 3 dedicated clients on a 9 data node cluster and got a 2x performance improvement. Moving the query coordination, network io (has to receive data from every shard), and combination of results (aggs and sorts) off of the nodes providing the results is very helpful. 

James



On Sat, May 30, 2015 at 9:11 AM, Xudong You <<a onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" href="javascript:" target="_blank" rel="nofollow" gdf-obfuscated-mailto="iWYZx3xT9UsJ">xudon...@...> wrote:
Thanks Nikolas,
How do you think about dedicated "client node" (the so called load balance node)? Any benefit of dedicated client node? Seems to me, round robin to data nodes is good enough.

On Friday, May 29, 2015 at 10:55:01 PM UTC+8, Nikolas Everett wrote:
Dedicated master nodes are super convenient if you have the it infrastructure to host them on shared machines because they are very low load and its useful to be able to restart the master nodes quickly. We don't have that kind of infrastructure and our cluster is pretty large and not having it has bitten us once or twice but its not a huge problem.



On Fri, May 29, 2015 at 10:44 AM, Xudong You <[hidden email]> wrote:
Right now we only need 4 ES nodes due to the small data volume, and all 4 nodes are master & data nodes.

Q1:
I am wondering in this case, is it necessary to have dedicated master and client node? Any benefit of having dedicated master node?

Some one said that dedicated master nodes (say, three master nodes) is helpful to avoid the split brain issue, but even we have NO dedicated master nodes, we can also avoid the split brain by setting the discovery.zen.minimum_master_nodes to a appropriate value.

Q2:
Similarly, is dedicated client node really necessary in our 4 nodes case? Any benefit of allocating dedicated client node?

Thanks!

--
Please update your bookmarks! We have moved to <a onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;" href="https://discuss.elastic.co/" target="_blank" rel="nofollow">https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" href="https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow">https://groups.google.com/d/msgid/elasticsearch/de7db788-a6d2-48c2-934b-bc5f7ae311a9%40googlegroups.com.
For more options, visit <a onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;" href="https://groups.google.com/d/optout" target="_blank" rel="nofollow">https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to <a onmousedown="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;" onclick="this.href='https://www.google.com/url?q\75https%3A%2F%2Fdiscuss.elastic.co%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFguF42tUvBArAl-xFnDHHIi5v0cA';return true;" href="https://discuss.elastic.co/" target="_blank" rel="nofollow">https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" href="javascript:" target="_blank" rel="nofollow" gdf-obfuscated-mailto="iWYZx3xT9UsJ">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" href="https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow">https://groups.google.com/d/msgid/elasticsearch/d61afae2-2e47-4b65-866b-5a55d28b84ea%40googlegroups.com.

For more options, visit <a onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;" href="https://groups.google.com/d/optout" target="_blank" rel="nofollow">https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/
---
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6a178f09-b94f-4400-9267-bec1f5278ba0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.