Mappings for "store" do not work

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Mappings for "store" do not work

John Chang
I have created mappings for my index to tell the service not to sore certain fields.  However, when I do a GET on the id of the document, I still the the value for that field.

Indexing a document, I declare the "body" field as:
            "body" : {
                "index_name": "body",
                "type": "string",
                "index": "analyzed",
                "store": "no"
            }

But when I do a search (http://localhost:9200/index0/document/2036e745da914b55b56027c2c8eaa021) the body is still visible.

I don't want to to keep the body for security reasons.  This is an issue for me for numerous fields across numerous document types I index.  What can i do to make these valuables searchable, but not stored and not readable, in Elastic Search?

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Mappings for "store" do not work

kimchy
Administrator
Do you mean that it is returned as part of the _source response? Fields by default are not stored, but the actual json document used to index is stored, which can be disabled by controlling the _source mapping.

-shay.banon

On Wed, Jul 21, 2010 at 2:55 AM, John Chang <[hidden email]> wrote:

I have created mappings for my index to tell the service not to sore certain
fields.  However, when I do a GET on the id of the document, I still the the
value for that field.

Indexing a document, I declare the "body" field as:
           "body" : {
               "index_name":   "body",
               "type": "string",
               "index":        "analyzed",
               "store":        "no"
           }

But when I do a search
(http://localhost:9200/index0/document/2036e745da914b55b56027c2c8eaa021) the
body is still visible.

I don't want to to keep the body for security reasons.  This is an issue for
me for numerous fields across numerous document types I index.  What can i
do to make these valuables searchable, but not stored and not readable, in
Elastic Search?

Thanks
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Mappings-for-store-do-not-work-tp982897p982897.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Mappings for "store" do not work

John Chang
How can I get at the fields I've mapped at "store" : "yes" when I've disabled the source?

You are right, it was the _source where I was seeing the stored json document, and I need to make sure I'm not holding on to that.  So, I used the "_source" mapping as you advised:

curl -XPUT http://localhost:9200/index0/document/_mapping -d '
{
    "document" : {
        "_source" : {"enabled" : false},

It gets rid of the source json, but now I can't get anything from the document other than the ID.  (I can get the id on the org.elasticsearch.search.SearchHit objects in  the org.elasticsearch.search.SearchHits list in org.elasticsearch.action.search.SearchResponse.)

I want to still be able to get at some of the fields (those I mapped as "store" : "yes").   Disabling the source, I of course now get nulls from these methods on org.elasticsearch.search.SearchHit:
  sourceAsMap() (what I had been calling before disabling source)
  getSource()
And I get empty maps back from these methods on org.elasticsearch.search.SearchHit:
  getFields()
  getHighlightFields()
  fields()

Thanks,
John
Reply | Threaded
Open this post in threaded view
|

Re: Mappings for "store" do not work

Lukáš Vlček
Hi John,

I think what you are after now can be found here:

Check the Request body - Fields:


{
   
"fields" : ["user", "postDate"],
   
"query" : {
       
"term" : { "user" : "kimchy" }
   
}
}
Regards,
Lukas

On Wed, Jul 21, 2010 at 7:23 PM, John Chang <[hidden email]> wrote:

How can I get at the fields I've mapped at "store" : "yes" when I've disabled
the source?

You are right, it was the _source where I was seeing the stored json
document, and I need to make sure I'm not holding on to that.  So, I used
the "_source" mapping as you advised:

curl -XPUT http://localhost:9200/index0/document/_mapping -d '
{
   "document" : {
       "_source" : {"enabled" : false},

It gets rid of the source json, but now I can't get anything from the
document other than the ID.  (I can get the id on the
org.elasticsearch.search.SearchHit objects in  the
org.elasticsearch.search.SearchHits list in
org.elasticsearch.action.search.SearchResponse.)

I want to still be able to get at some of the fields (those I mapped as
"store" : "yes").   Disabling the source, I of course now get nulls from
these methods on org.elasticsearch.search.SearchHit:
 sourceAsMap() (what I had been calling before disabling source)
 getSource()
And I get empty maps back from these methods on
org.elasticsearch.search.SearchHit:
 getFields()
 getHighlightFields()
 fields()

Thanks,
John
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Mappings-for-store-do-not-work-tp982897p984900.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.