"Idle" node traffic

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

"Idle" node traffic

Glen Smith
In a cluster where there is no client activity happening (no index or search requests), how much data, roughly, is being sent between the nodes?
I was able to find a nice blog post from Njal Karevoll at found.no that explained all the transport channels per node, but nothing about the steady-state volume of data.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: "Idle" node traffic

Nikolas Everett
Its not exactly what you asked for, but its something.  I imagine the activity on empty nodes when there is no traffic would be lower but I'm not sure by how much.

Nik


On Fri, Jul 18, 2014 at 10:01 AM, Glen Smith <[hidden email]> wrote:
In a cluster where there is no client activity happening (no index or search requests), how much data, roughly, is being sent between the nodes?
I was able to find a nice blog post from Njal Karevoll at found.no that explained all the transport channels per node, but nothing about the steady-state volume of data.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0cP8Qx8UbupfLtmMfEZaUgUGC0-%2B35XB3W6cLLW9RiGg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: "Idle" node traffic

Glen Smith
Thank you very much for the links!

I'm trying to infer what I can from them (looks like there are 19 nodes in the cluster, and this single cluster is averaging about 165kBytes/sec sent over the last hour.

I'm still hoping there will be a more formal answer detailing the node-to-node communications, how it might geometrically increase with the number of nodes (since, AFAICT, each node pair is directly connected), whether it changes based on the number of indices or shards, etc.


On Friday, July 18, 2014 10:36:25 AM UTC-4, Nikolas Everett wrote:
This node is empty but running in a cluster serving traffic:
<a href="http://ganglia.wikimedia.org/latest/?c=Elasticsearch%20cluster%20eqiad&amp;h=elastic1017.eqiad.wmnet&amp;m=cpu_report&amp;r=hour&amp;s=by%20name&amp;hc=4&amp;mc=2" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fganglia.wikimedia.org%2Flatest%2F%3Fc%3DElasticsearch%2520cluster%2520eqiad%26h%3Delastic1017.eqiad.wmnet%26m%3Dcpu_report%26r%3Dhour%26s%3Dby%2520name%26hc%3D4%26mc%3D2\46sa\75D\46sntz\0751\46usg\75AFQjCNH_if6CEypCBocj-BQsJ8GtkSLV1A';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fganglia.wikimedia.org%2Flatest%2F%3Fc%3DElasticsearch%2520cluster%2520eqiad%26h%3Delastic1017.eqiad.wmnet%26m%3Dcpu_report%26r%3Dhour%26s%3Dby%2520name%26hc%3D4%26mc%3D2\46sa\75D\46sntz\0751\46usg\75AFQjCNH_if6CEypCBocj-BQsJ8GtkSLV1A';return true;">http://ganglia.wikimedia.org/latest/?c=Elasticsearch%20cluster%20eqiad&h=elastic1017.eqiad.wmnet&m=cpu_report&r=hour&s=by%20name&hc=4&mc=2
Here is the rest of the cluster:
<a href="http://ganglia.wikimedia.org/latest/?c=Elasticsearch%20cluster%20eqiad&amp;m=cpu_report&amp;r=hour&amp;s=by%20name&amp;hc=4&amp;mc=2" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fganglia.wikimedia.org%2Flatest%2F%3Fc%3DElasticsearch%2520cluster%2520eqiad%26m%3Dcpu_report%26r%3Dhour%26s%3Dby%2520name%26hc%3D4%26mc%3D2\46sa\75D\46sntz\0751\46usg\75AFQjCNEYGkyXYdbFNfdK-yprCPdfjmW9FA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fganglia.wikimedia.org%2Flatest%2F%3Fc%3DElasticsearch%2520cluster%2520eqiad%26m%3Dcpu_report%26r%3Dhour%26s%3Dby%2520name%26hc%3D4%26mc%3D2\46sa\75D\46sntz\0751\46usg\75AFQjCNEYGkyXYdbFNfdK-yprCPdfjmW9FA';return true;">http://ganglia.wikimedia.org/latest/?c=Elasticsearch%20cluster%20eqiad&m=cpu_report&r=hour&s=by%20name&hc=4&mc=2

Its not exactly what you asked for, but its something.  I imagine the activity on empty nodes when there is no traffic would be lower but I'm not sure by how much.

Nik


On Fri, Jul 18, 2014 at 10:01 AM, Glen Smith <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="gdG5tNP0MM4J" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">gl...@...> wrote:
In a cluster where there is no client activity happening (no index or search requests), how much data, roughly, is being sent between the nodes?
I was able to find a nice blog post from Njal Karevoll at <a href="http://found.no" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Ffound.no\46sa\75D\46sntz\0751\46usg\75AFQjCNE4HWMNqKOsppe3x3uEokp2u4bn5g';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Ffound.no\46sa\75D\46sntz\0751\46usg\75AFQjCNE4HWMNqKOsppe3x3uEokp2u4bn5g';return true;">found.no that explained all the transport channels per node, but nothing about the steady-state volume of data.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="gdG5tNP0MM4J" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;">https://groups.google.com/d/msgid/elasticsearch/caf2799b-82d2-4a28-8671-60f0e9f96137%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20fb79c0-75f5-418a-ab47-3cc7236fc10d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.