ES Document Update Issue MR

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ES Document Update Issue MR

Tauqeer Ul Islam
This post has NOT been accepted by the mailing list yet.
This post was updated on .
I am trying to use ES hadoop MR to update documents using following properties:
  • es.write.operation=update
  • es.mapping.id=_metadata._id
  • es.read.metadata.field=metadata
  • es.read.metadata=true
On each document write function call, I am getting this type of exception.
2015-08-20 16:38:34,895 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: [MapWritableFieldExtractor for field [_metadata._id]] cannot extract value from entity [class java.lang.String] | instance [{doc_id=AU9Jtv-wfhVNYAhE2vWb, test_filed=tempData, _metadata={_index=sa-analytics-new-2, _type=logs, _id=AU9KfBg9fhVNYAhE21El, _score=0.0}, Temp_id=AU9KfBg9fhVNYAhE21El}] at org.elasticsearch.hadoop.serialization.bulk.AbstractBulkFactory$FieldWriter.write(AbstractBulkFactory.java:97) at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.writeTemplate(TemplatedBulk.java:80) at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:56) at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:148) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:151) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:635) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) I think there are some issues in es.mapping.id value. May be it is not allowed to access fields like i have given here. Any help in this will be appreciated Thanks
Reply | Threaded
Open this post in threaded view
|

Re: ES Document Update Issue MR

imad
This post has NOT been accepted by the mailing list yet.
Post your questions @ https://discuss.elastic.co/. Elasticsearch Users mailing list is no longer maintained.

On Thu, Aug 20, 2015 at 11:02 AM, Tauqeer Ul Islam [via Elasticsearch Users] <[hidden email]> wrote:
I am trying to use ES hadoop MR to update documents using following properties: es.write.operation=update es.mapping.id=_metadata._id es.read.metadata.field=metadata es.read.metadata=true On each document write function call, I am getting this type of exception. 2015-08-20 16:38:34,895 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: [MapWritableFieldExtractor for field [_metadata._id]] cannot extract value from entity [class java.lang.String] | instance [{doc_id=AU9Jtv-wfhVNYAhE2vWb, test_filed=tempData, _metadata={_index=sa-analytics-new-2, _type=logs, _id=AU9KfBg9fhVNYAhE21El, _score=0.0}, Temp_id=AU9KfBg9fhVNYAhE21El}] at org.elasticsearch.hadoop.serialization.bulk.AbstractBulkFactory$FieldWriter.write(AbstractBulkFactory.java:97) at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.writeTemplate(TemplatedBulk.java:80) at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:56) at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:148) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:151) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:635) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) I think there are some issues in es.mapping.id value. May be it is not allowed to access fields like i have given here. Any help in this will be appreciated Thanks


If you reply to this email, your message will be added to the discussion below:
http://elasticsearch-users.115913.n3.nabble.com/ES-Document-Update-Issue-MR-tp4075214.html
To start a new topic under Elasticsearch Users, email [hidden email]
To unsubscribe from Elasticsearch Users, click here.
NAML