Update mapping - what is safe to ignore?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Update mapping - what is safe to ignore?

Ivan Brusic
Using ES 0.16.2

Due to a change in the source content that I am indexing, I need to update a field from a long to string.  

The original mapping is dynamic, no explicit mapping was ever provided:
"event": {
    "dynamic": "true", 
    "event_id": { "type": "long" },
    // other fields ...
}

The field is never used and I do not care of "legacy" documents having the correct type.  Looking to change this value to a simple not_analyzed string:
"event_id": { "type": "string", "index": "not_analyzed" }

Of course, an update mapping between types does not work:
{"error":"MergeMappingException[Merge failed with failures {[mapper [event.event_id] of different type, current_type [long], merged_type [string]]}]","status":400}

At this point I set ignore_conflicts=true and executed the update mapping (via curl).  I received an acknowledgment, but the mapping is not executed.
{"ok":true,"acknowledged":true}

No errors appear in the log. I used used (abused?) ignore_conflicts in the past with success, but I am not surprised it does not work in every case.  What are the cases were an update to a mapping simply is not possible?

My next question is for reindexing recipes, but that is for a different thread.

Cheers,

Ivan
Reply | Threaded
Open this post in threaded view
|

Re: Update mapping - what is safe to ignore?

Karussell
updates to the mapping in your case will cause a lot of trouble.
change the mapping and create a new index. then reindex and switch the
alias and optionally delete the old index.

Otherwise strange things can happen ...


On 22 Jul., 15:06, Ivan Brusic <[hidden email]> wrote:

> Using ES 0.16.2
>
> Due to a change in the source content that I am indexing, I need to update a
> field from a long to string.
>
> The original mapping is dynamic, no explicit mapping was ever provided:
> "event": {
>     "dynamic": "true",
>     "event_id": { "type": "long" },
>     // other fields ...
>
> }
>
> The field is never used and I do not care of "legacy" documents having the
> correct type.  Looking to change this value to a simple not_analyzed string:
> "event_id": { "type": "string", "index": "not_analyzed" }
>
> Of course, an update mapping between types does not work:
> {"error":"MergeMappingException[Merge failed with failures {[mapper
> [event.event_id] of different type, current_type [long], merged_type
> [string]]}]","status":400}
>
> At this point I set ignore_conflicts=true and executed the update mapping
> (via curl).  I received an acknowledgment, but the mapping is not executed.
> {"ok":true,"acknowledged":true}
>
> No errors appear in the log. I used used (abused?) ignore_conflicts in the
> past with success, but I am not surprised it does not work in every case.
>  What are the cases were an update to a mapping simply is not possible?
>
> My next question is for reindexing recipes, but that is for a different
> thread.
>
> Cheers,
>
> Ivan
Reply | Threaded
Open this post in threaded view
|

Re: Update mapping - what is safe to ignore?

kimchy
Administrator
In reply to this post by Ivan Brusic
You can't change the type of a field mapped, even when ignore_conflicts is set to true. Ignore conflicts simply does that, ignore the conflicts and applies what it can, it does not apply conflicts.

On Fri, Jul 22, 2011 at 4:06 PM, Ivan Brusic <[hidden email]> wrote:
Using ES 0.16.2

Due to a change in the source content that I am indexing, I need to update a field from a long to string.  

The original mapping is dynamic, no explicit mapping was ever provided:
"event": {
    "dynamic": "true", 
    "event_id": { "type": "long" },
    // other fields ...
}

The field is never used and I do not care of "legacy" documents having the correct type.  Looking to change this value to a simple not_analyzed string:
"event_id": { "type": "string", "index": "not_analyzed" }

Of course, an update mapping between types does not work:
{"error":"MergeMappingException[Merge failed with failures {[mapper [event.event_id] of different type, current_type [long], merged_type [string]]}]","status":400}

At this point I set ignore_conflicts=true and executed the update mapping (via curl).  I received an acknowledgment, but the mapping is not executed.
{"ok":true,"acknowledged":true}

No errors appear in the log. I used used (abused?) ignore_conflicts in the past with success, but I am not surprised it does not work in every case.  What are the cases were an update to a mapping simply is not possible?

My next question is for reindexing recipes, but that is for a different thread.

Cheers,

Ivan

Reply | Threaded
Open this post in threaded view
|

Re: Update mapping - what is safe to ignore?

Ivan Brusic
I guess the question I really meant to ask is what updates are/are-not possible?  Change of type is not permitted.  What else?

On Sat, Jul 23, 2011 at 10:16 PM, Shay Banon <[hidden email]> wrote:
You can't change the type of a field mapped, even when ignore_conflicts is set to true. Ignore conflicts simply does that, ignore the conflicts and applies what it can, it does not apply conflicts.


On Fri, Jul 22, 2011 at 4:06 PM, Ivan Brusic <[hidden email]> wrote:
Using ES 0.16.2

Due to a change in the source content that I am indexing, I need to update a field from a long to string.  

The original mapping is dynamic, no explicit mapping was ever provided:
"event": {
    "dynamic": "true", 
    "event_id": { "type": "long" },
    // other fields ...
}

The field is never used and I do not care of "legacy" documents having the correct type.  Looking to change this value to a simple not_analyzed string:
"event_id": { "type": "string", "index": "not_analyzed" }

Of course, an update mapping between types does not work:
{"error":"MergeMappingException[Merge failed with failures {[mapper [event.event_id] of different type, current_type [long], merged_type [string]]}]","status":400}

At this point I set ignore_conflicts=true and executed the update mapping (via curl).  I received an acknowledgment, but the mapping is not executed.
{"ok":true,"acknowledged":true}

No errors appear in the log. I used used (abused?) ignore_conflicts in the past with success, but I am not surprised it does not work in every case.  What are the cases were an update to a mapping simply is not possible?

My next question is for reindexing recipes, but that is for a different thread.

Cheers,

Ivan


Reply | Threaded
Open this post in threaded view
|

Re: Update mapping - what is safe to ignore?

kimchy
Administrator
Yea, this needs to be documented, open an issue, I'll doc it.

On Tue, Jul 26, 2011 at 12:48 AM, Ivan Brusic <[hidden email]> wrote:
I guess the question I really meant to ask is what updates are/are-not possible?  Change of type is not permitted.  What else?

On Sat, Jul 23, 2011 at 10:16 PM, Shay Banon <[hidden email]> wrote:
You can't change the type of a field mapped, even when ignore_conflicts is set to true. Ignore conflicts simply does that, ignore the conflicts and applies what it can, it does not apply conflicts.


On Fri, Jul 22, 2011 at 4:06 PM, Ivan Brusic <[hidden email]> wrote:
Using ES 0.16.2

Due to a change in the source content that I am indexing, I need to update a field from a long to string.  

The original mapping is dynamic, no explicit mapping was ever provided:
"event": {
    "dynamic": "true", 
    "event_id": { "type": "long" },
    // other fields ...
}

The field is never used and I do not care of "legacy" documents having the correct type.  Looking to change this value to a simple not_analyzed string:
"event_id": { "type": "string", "index": "not_analyzed" }

Of course, an update mapping between types does not work:
{"error":"MergeMappingException[Merge failed with failures {[mapper [event.event_id] of different type, current_type [long], merged_type [string]]}]","status":400}

At this point I set ignore_conflicts=true and executed the update mapping (via curl).  I received an acknowledgment, but the mapping is not executed.
{"ok":true,"acknowledged":true}

No errors appear in the log. I used used (abused?) ignore_conflicts in the past with success, but I am not surprised it does not work in every case.  What are the cases were an update to a mapping simply is not possible?

My next question is for reindexing recipes, but that is for a different thread.

Cheers,

Ivan



Reply | Threaded
Open this post in threaded view
|

Re: Update mapping - what is safe to ignore?

Ivan Brusic
Done: https://github.com/elasticsearch/elasticsearch/issues/1166

Obviously, a very low priority.  I can probably get the information myself by looking at the source code.

Thanks,

Ivan

On Tue, Jul 26, 2011 at 12:55 AM, Shay Banon <[hidden email]> wrote:
Yea, this needs to be documented, open an issue, I'll doc it.


On Tue, Jul 26, 2011 at 12:48 AM, Ivan Brusic <[hidden email]> wrote:
I guess the question I really meant to ask is what updates are/are-not possible?  Change of type is not permitted.  What else?

On Sat, Jul 23, 2011 at 10:16 PM, Shay Banon <[hidden email]> wrote:
You can't change the type of a field mapped, even when ignore_conflicts is set to true. Ignore conflicts simply does that, ignore the conflicts and applies what it can, it does not apply conflicts.


On Fri, Jul 22, 2011 at 4:06 PM, Ivan Brusic <[hidden email]> wrote:
Using ES 0.16.2

Due to a change in the source content that I am indexing, I need to update a field from a long to string.  

The original mapping is dynamic, no explicit mapping was ever provided:
"event": {
    "dynamic": "true", 
    "event_id": { "type": "long" },
    // other fields ...
}

The field is never used and I do not care of "legacy" documents having the correct type.  Looking to change this value to a simple not_analyzed string:
"event_id": { "type": "string", "index": "not_analyzed" }

Of course, an update mapping between types does not work:
{"error":"MergeMappingException[Merge failed with failures {[mapper [event.event_id] of different type, current_type [long], merged_type [string]]}]","status":400}

At this point I set ignore_conflicts=true and executed the update mapping (via curl).  I received an acknowledgment, but the mapping is not executed.
{"ok":true,"acknowledged":true}

No errors appear in the log. I used used (abused?) ignore_conflicts in the past with success, but I am not surprised it does not work in every case.  What are the cases were an update to a mapping simply is not possible?

My next question is for reindexing recipes, but that is for a different thread.

Cheers,

Ivan