Doubt on updating mappings

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Doubt on updating mappings

Mahendra M
Hi,

Had a few queries on using mappings. Mostly noob questions. I checked
the docs, but I was unable to find the answers.

I noticed this behaviour while testing ES.
* On a particular index, I have 100 documents indexed.
* After this, I update the mapping ( updated include_in_all & store
for a few fields )
* After this, I try a search query for _all fields and also with the
'fields' argument
* I noticed that my mapping update has not taken effect.
* Now, I re-index the 100 documents and my queries work with the new mapping.

Am I doing the thing wrong or is this how it is designed ? As in, if
mapping is updated, then all documents have to be re-indexed for the
new mapping to be applicable on them.

On a similar topic, does this apply to index configuration also.
Say I have an index with 2 shards and 2 replicas each. I have X
documents indexed. If I re-configure the index with 3 shards and 4
replicas, will things be taken care of automatically ? Or should I
re-index the documents again ?

PS: Elastic Search is really a wonderful tool :-)

Regards,
Mahendra

http://twitter.com/mahendra
Reply | Threaded
Open this post in threaded view
|

Re: Doubt on updating mappings

kimchy
Administrator
When updating mappings, you might have conflicts. Conflicts mean that you are trying to set something that can't be set. By default in 0.6, conflicts are ignored. I changed it to not be ignored in upcoming 0.7 so things will be more obvious. You can always control that using the ignore_conflicts parameter.

In your case, trying to change some settings on an existing mapper, it is considered as a conflict. In the future,  some settings might be applied dynamically, though without reindexing, just for new data indexed.

Note, buy the way, that you can upgrade an existing single mapping into a multi_field mapping, which is nice. Then, you can add more mapping on that multi_field mapping (its not considered a conflict). But, this will only apply to new data indexed.

cheers,
shay.banon

On Mon, May 3, 2010 at 10:53 AM, Mahendra M <[hidden email]> wrote:
Hi,

Had a few queries on using mappings. Mostly noob questions. I checked
the docs, but I was unable to find the answers.

I noticed this behaviour while testing ES.
* On a particular index, I have 100 documents indexed.
* After this, I update the mapping ( updated include_in_all & store
for a few fields )
* After this, I try a search query for _all fields and also with the
'fields' argument
* I noticed that my mapping update has not taken effect.
* Now, I re-index the 100 documents and my queries work with the new mapping.

Am I doing the thing wrong or is this how it is designed ? As in, if
mapping is updated, then all documents have to be re-indexed for the
new mapping to be applicable on them.

On a similar topic, does this apply to index configuration also.
Say I have an index with 2 shards and 2 replicas each. I have X
documents indexed. If I re-configure the index with 3 shards and 4
replicas, will things be taken care of automatically ? Or should I
re-index the documents again ?

PS: Elastic Search is really a wonderful tool :-)

Regards,
Mahendra

http://twitter.com/mahendra

Reply | Threaded
Open this post in threaded view
|

Re: Doubt on updating mappings

Mahendra M
On Mon, May 3, 2010 at 1:58 PM, Shay Banon <[hidden email]> wrote:
> In your case, trying to change some settings on an existing mapper, it is
> considered as a conflict. In the future,  some settings might be applied
> dynamically, though without reindexing, just for new data indexed.

Sounds good. So, if I have to change the mapping, I will re-index the
data. Though in production, this will be a very very rare scenario.

Thanks,
Mahendra
Reply | Threaded
Open this post in threaded view
|

Re: Doubt on updating mappings

mooky
Since elastic stores the raw data, it's possible the re-indexing could  
be done entirely internally, no?

-N

On 3 May 2010, at 14:30, Mahendra M <[hidden email]> wrote:

> On Mon, May 3, 2010 at 1:58 PM, Shay Banon <[hidden email]
> > wrote:
>> In your case, trying to change some settings on an existing mapper,  
>> it is
>> considered as a conflict. In the future,  some settings might be  
>> applied
>> dynamically, though without reindexing, just for new data indexed.
>
> Sounds good. So, if I have to change the mapping, I will re-index the
> data. Though in production, this will be a very very rare scenario.
>
> Thanks,
> Mahendra
Reply | Threaded
Open this post in threaded view
|

Re: Doubt on updating mappings

kimchy
Administrator
Yep, if storing the _source is enabled, then there can be an "index_to" API that will completely reindex one index into another.

On Mon, May 3, 2010 at 7:10 PM, Nick Minutello <[hidden email]> wrote:
Since elastic stores the raw data, it's possible the re-indexing could be done entirely internally, no?

-N


On 3 May 2010, at 14:30, Mahendra M <[hidden email]> wrote:

On Mon, May 3, 2010 at 1:58 PM, Shay Banon <[hidden email]> wrote:
In your case, trying to change some settings on an existing mapper, it is
considered as a conflict. In the future,  some settings might be applied
dynamically, though without reindexing, just for new data indexed.

Sounds good. So, if I have to change the mapping, I will re-index the
data. Though in production, this will be a very very rare scenario.

Thanks,
Mahendra