jdbc river re-indexing after each start of server?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

jdbc river re-indexing after each start of server?

Abid Hussain
Hi all,

I configured a river for one-time indexing like:
PUT /_river/my_river/_meta
{
      "type": "jdbc",
      "jdbc": {
         "driver": "com.mysql.jdbc.Driver",
         "url": "jdbc:mysql://192.168.110.180:3306/my_db",
         "user": "user",
         "password": "password",
         "index": "my_idx",
         "type": "order",
         "sql": "SELECT o.order_id AS _id, o.order_name FROM orders o"
      }
}


After performing the PUT, the data is being indexed. When finished, I stop the server and start it again. The CPU load now is remarkably high. Looking at  /_river/jdbc/*/_state it tells me that last_active_end is null.

So, does the restart of elasicsearch trigger a complete reindexing? If so, how can I avoid this - e.g by using timestamps?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jdbc river re-indexing after each start of server?

joergprante@gmail.com
You should delete the river instance after usage. Otherwise, the river is executed each time the node starts.

Jörg

On Mon, Jan 26, 2015 at 1:09 PM, Abid Hussain <[hidden email]> wrote:
Hi all,

I configured a river for one-time indexing like:
PUT /_river/my_river/_meta
{
      "type": "jdbc",
      "jdbc": {
         "driver": "com.mysql.jdbc.Driver",
         "url": "jdbc:mysql://192.168.110.180:3306/my_db",
         "user": "user",
         "password": "password",
         "index": "my_idx",
         "type": "order",
         "sql": "SELECT o.order_id AS _id, o.order_name FROM orders o"
      }
}


After performing the PUT, the data is being indexed. When finished, I stop the server and start it again. The CPU load now is remarkably high. Looking at  /_river/jdbc/*/_state it tells me that last_active_end is null.

So, does the restart of elasicsearch trigger a complete reindexing? If so, how can I avoid this - e.g by using timestamps?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE9J_TvKbE7%2BWO7_DByD%3DoQ1M293BOxLoX_AchyeMMJJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: jdbc river re-indexing after each start of server?

Abid Hussain
Thanks for you quick help!

Am Montag, 26. Januar 2015 13:15:26 UTC+1 schrieb Jörg Prante:
You should delete the river instance after usage. Otherwise, the river is executed each time the node starts.

Jörg

On Mon, Jan 26, 2015 at 1:09 PM, Abid Hussain <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="EifwE3m4tqcJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">hus...@...> wrote:
Hi all,

I configured a river for one-time indexing like:
PUT /_river/my_river/_meta
{
      "type": "jdbc",
      "jdbc": {
         "driver": "com.mysql.jdbc.Driver",
         "url": "jdbc:mysql://<a href="http://192.168.110.180:3306/my_db" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2F192.168.110.180%3A3306%2Fmy_db\46sa\75D\46sntz\0751\46usg\75AFQjCNEJNaJxYMobrMHPDpgSZlv41oZVMw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2F192.168.110.180%3A3306%2Fmy_db\46sa\75D\46sntz\0751\46usg\75AFQjCNEJNaJxYMobrMHPDpgSZlv41oZVMw';return true;">192.168.110.180:3306/my_db",
         "user": "user",
         "password": "password",
         "index": "my_idx",
         "type": "order",
         "sql": "SELECT o.order_id AS _id, o.order_name FROM orders o"
      }
}


After performing the PUT, the data is being indexed. When finished, I stop the server and start it again. The CPU load now is remarkably high. Looking at  /_river/jdbc/*/_state it tells me that last_active_end is null.

So, does the restart of elasicsearch trigger a complete reindexing? If so, how can I avoid this - e.g by using timestamps?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="EifwE3m4tqcJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;">https://groups.google.com/d/msgid/elasticsearch/b006c70b-93f7-48a9-b423-136d288ef7f3%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0be9e039-87e2-44fa-8952-67711e83cfe8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.