[ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

dadoonet

Heya,

We are pleased to announce the release of the Elasticsearch Mapper Attachment plugin, version 2.3.0

The mapper attachments plugin adds the attachment type to Elasticsearch using Apache Tika..

Release Notes - Version 2.3.0

Update

  • [78] - Log tika exceptions
  • [76] - Update to Lucene 4.9.0
  • [73] - deprecate `content` by `_content`
  • [69] - Remove deprecated `language` forced field
  • [67] - Update to elasticsearch 1.3.0

Issues, Pull requests, Feature requests are warmly welcome on elasticsearch-mapper-attachments project repository!

For questions or comments around this plugin, feel free to use elasticsearch mailing list!

Enjoy,

- The Elasticsearch team

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53d2de83.4624b40a.1097.0157SMTPIN_ADDED_MISSING%40gmr-mx.google.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

Prashant Agrawal
Hi dadoonet,

Currently we were using Elasticsearch 1.3.0 and supported Mapper Plugin for the same. Now we are moving Elasticsearch 1.5.2 so we need to have the updated mapper att plugin as well.

As in the latest one content is deprecated by _content, so in our application we need to change the content to _content while retrieving the document. So here my question is like is there way to re index the Elasticsearch data so that older records can also be returned using _content instead of content (any way like changing some conf or doing re indexing or something else)
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

dadoonet
Just a note before answering, the change is about the document you send to elasticsearch. Instead of sending:

{
  "file": {
    "content": "VGhpcyBpcyBhbiBlbGFzdGljc2VhcmNoIG1hcHBlciBhdHRhY2htZW50IHRlc3Qu",
    "_name": "myfilename.txt"
  }
}

You now send

{
  "file": {
    "_content": "VGhpcyBpcyBhbiBlbGFzdGljc2VhcmNoIG1hcHBlciBhdHRhY2htZW50IHRlc3Qu",
    "_name": "myfilename.txt"
  }
}

It’s not related to generated field names at index time.


That said, I think you should better handle that on client side unless you have a few docs to reindex.
That answers to your question, yes you have to reindex if you want to use the new format instead of the old one.


Does it help?

-- 
David Pilato - Developer | Evangelist 





Le 30 avr. 2015 à 12:12, Prashant Agrawal <[hidden email]> a écrit :

Hi dadoonet,

Currently we were using Elasticsearch 1.3.0 and supported Mapper Plugin for
the same. Now we are moving Elasticsearch 1.5.2 so we need to have the
updated mapper att plugin as well.

As in the latest one content is deprecated by _content, so in our
application we need to change the content to _content while retrieving the
document. So here my question is like is there way to re index the
Elasticsearch data so that older records can also be returned using _content
instead of content (any way like changing some conf or doing re indexing or
something else)



--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/ANN-Elasticsearch-Mapper-Attachment-plugin-2-3-0-released-tp4060656p4074364.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1430388737162-4074364.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2E348C66-DC16-421C-B905-5F6BA3C74332%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

Prashant Agrawal
Hi ,
Thanks for a quick response,

Yes , I am making a change content to _content as an index request.

Here my worry is like our application is already deployed , so with new changes being done I will not be able to fetch the older records using the same application as older records will be returned with "content" (in source) and new records will be returned with "_content".

So what could be the best way to handle this using elasticsearch.

~Prashant
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

dadoonet
So why not doing that in your application?

If you look into _source.file.content and _source.file._content, older docs will have BASE64 content in content and null in _content and the opposite for newer docs.

Do I miss anything?

-- 
David Pilato - Developer | Evangelist 





Le 30 avr. 2015 à 12:33, Prashant Agrawal <[hidden email]> a écrit :

Hi ,
Thanks for a quick response,

Yes , I am making a change content to _content as an index request.

Here my worry is like our application is already deployed , so with new
changes being done I will not be able to fetch the older records using the
same application as older records will be returned with "content" (in
source) and new records will be returned with "_content".

So what could be the best way to handle this using elasticsearch.

~Prashant



--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/ANN-Elasticsearch-Mapper-Attachment-plugin-2-3-0-released-tp4060656p4074367.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1430390026413-4074367.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9FA37BF2-A60A-42CF-92BD-01DC5FDA90D0%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

Prashant Agrawal
Yes, Though that can be handled as an exception in our application.

But I was just looking for as if any thing is there in ES which can re index old document in parallel, and make the content present in older records under "_content" only instead of "content".


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

dadoonet
reindex is not a feature in elasticsearch nowadays.

How many documents you have to reindex?
IIRC your case you could have a lot, right?

-- 
David Pilato - Developer | Evangelist 





Le 30 avr. 2015 à 12:47, Prashant Agrawal <[hidden email]> a écrit :

Yes, Though that can be handled as an exception in our application.

But I was just looking for as if any thing is there in ES which can re index
old document in parallel, and make the content present in older records
under "_content" only instead of "content".






--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/ANN-Elasticsearch-Mapper-Attachment-plugin-2-3-0-released-tp4060656p4074370.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1430390860202-4074370.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1500B0C7-4C08-4A28-BD31-BB8C40476600%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

Prashant Agrawal
Ok, So handling at the client side would be the best solution for the same.

Yes, we do have records in TB's which may lead to billions of documents to be reindexed or handled differently.