how to migrate old elastic search index to new version of elastic search?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

how to migrate old elastic search index to new version of elastic search?

Sumit Guptaa
hi all,


Essentially I cannot find documents or resources that explains the procedure of upgrading a running Elastic search instance into the current version.


Please help me out in a few scenarios:-

1 If I am running an Elastic search instance in a single server, how do I upgrade the instance and not lose data?

2 If I am running multiple Elasticsearch instances in a number of servers, how do I keep my operations running, while I upgrade my Elasticsearch instances without losing data?

if there are proper procedures or explanations on this it will greatly help my understanding and work. Thanks!

Regards,
Sumit Gupta
Reply | Threaded
Open this post in threaded view
|

Re: how to migrate old elastic search index to new version of elastic search?

sujoysett
Hi,

I have recently upgraded my ES indices from elasticsearch 0.18.6 to 0.19.2, and the procedure followed was:
  • Stop indexing operations on the indices, run refresh and flush on the indices (at cluster level)
  • Stop all the nodes.
  • Upgrade all the nodes to next version. (In my case data location was separate, so no copying of data needed, only in the new version config file I had to mention the existing data location.)
  • Start the nodes (new version).
The above steps will work for case 1. But there will be a downtime.

For case 2, a more complex procedure is available, but I have NOT tried it personally at production level. I had success with it at Development stage.
  • First ensure replicas of all the shards exists in all the nodes, change replica count if necessary for this and wait until green.
  • Of the multiple available nodes, shut down some, and change the cluster name in configuration file as well as in local data folder.
  • This will prevent these nodes from interfering with existing active ones due to mismatch in cluster name. Proceed with upgrading these set of nodes while the others serve your application.
  • Once you make these upgraded nodes running, shut down the rest and proceed for upgrading them also.
These steps worked for us as we were using pure http client (no java API, so changing cluster name did not create problem), and using commons-httpclient-failover for the requests can help in switching over automatically. However, most of our operations were search based, and it created no problem, there might be data mismatch if indexing operations remain active during this activity, I am not sure.

Please note that the second solution posted above is only a trial workout, worked in my case, and not a general solution.


On Monday, May 28, 2012 10:42:38 AM UTC+5:30, Sumit Gupta wrote:
hi all,


Essentially I cannot find documents or resources that explains the procedure
of upgrading a running Elastic search instance into the current version.


Please help me out in a few scenarios:-

1 If I am running an Elastic search instance in a single server, how do I
upgrade the instance and not lose data?

2 If I am running multiple Elasticsearch instances in a number of servers,
how do I keep my operations running, while I upgrade my Elasticsearch
instances without losing data?

if there are proper procedures or explanations on this it will greatly help
my understanding and work. Thanks!

Regards,
Sumit Gupta

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to migrate old elastic search index to new version of elastic search?

Sumit Guptaa
In reply to this post by Sumit Guptaa
hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop all nodes,and i want to ask one more things  if we migrate our data from Elastic Search 0.18.7 to elastic search 0.19.0 then the data that is store in elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta
Reply | Threaded
Open this post in threaded view
|

Re: how to migrate old elastic search index to new version of elastic search?

pulkitsinghal
I'll try to bridge the gap: nodes means servers

On Monday, May 28, 2012 4:02:08 AM UTC-5, Sumit Gupta wrote:
hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop all
nodes,and i want to ask one more things  if we migrate our data from Elastic
Search 0.18.7 to elastic search 0.19.0 then the data that is store in
elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412p4018426.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to migrate old elastic search index to new version of elastic search?

sujoysett
Yes, by nodes I meant running instances of elasticsearch.
And for the second question, yes, that is what the term migration means. Keeping your existing data as it is, but using the same on a new elasticsearch version.

Sujoy.

On Monday, May 28, 2012 6:23:56 PM UTC+5:30, pulkitsinghal wrote:
I'll try to bridge the gap: nodes means servers

On Monday, May 28, 2012 4:02:08 AM UTC-5, Sumit Gupta wrote:
hi sujoysett

can you tell me what is the means of stop all nodes and how we can stop all
nodes,and i want to ask one more things  if we migrate our data from Elastic
Search 0.18.7 to elastic search 0.19.0 then the data that is store in
elastic search 0.18.7 is store or not?

thanx

Regards,
Sumit Gupta

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/how-to-migrate-old-elastic-search-index-to-new-version-of-elastic-search-tp4018412p4018426.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to migrate old elastic search index to new version of elastic search?

Sumit Guptaa
thanx sujoysett