Querying specific fields in multiple mappings

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Querying specific fields in multiple mappings

arien
Hi all;
I have some questions when it comes to querying specific fields in
multiple mappings provided that all mappings belongs to same index.say
"test" is the mapping and "testmap1","testmap2","testmap3" are the
mappings of an index.
In query i dont want to specify any mapping name just simply wants to
put searchable columns there.
And the second thing; is it possible to register mapping specific
query against particular index,if so please provide some inputs.

Thanks;
Arien
Reply | Threaded
Open this post in threaded view
|

Re: Querying specific fields in multiple mappings

kimchy
Administrator
I did not understand the question properly... . You can specify "field_name", but you can also specify "type1.field_name", which will automatically wrap the query to be filtered by the type.

On Tue, Nov 29, 2011 at 11:53 AM, arien <[hidden email]> wrote:
Hi all;
I have some questions when it comes to querying specific fields in
multiple mappings provided that all mappings belongs to same index.say
"test" is the mapping and "testmap1","testmap2","testmap3" are the
mappings of an index.
In query i dont want to specify any mapping name just simply wants to
put searchable columns there.
And the second thing; is it possible to register mapping specific
query against particular index,if so please provide some inputs.

Thanks;
Arien

Reply | Threaded
Open this post in threaded view
|

Re: Querying specific fields in multiple mappings

arien
Hi shay,
Sorry for that i didn't frame my questions correctly.
In order the elaborate the scenario lets say i have an index "index"
with two mappings "mapping1","mapping2" and following are the records
in those;
============================================
curl -XPUT 'http://localhost:9200/index/mapping1/1' -d '{
    "user" : "kim",
    "post" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/2' -d '{
    "user" : "ajay",
    "post" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/3' -d '{
    "user" : "bhavani",
    "post" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}'
============================================
curl -XPUT 'http://localhost:9200/index/mapping2/1' -d '{
    "id" : "kim",
    "think" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/2' -d '{
    "id" : "kim",
    "think" : "2009-11-15T14:12:12"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/3' -d '{
    "id" : "kim",
    "think" : "2009-11-15T14:12:12"
}'
=============================================
and if i Search using query


curl "localhost:9200/index/_search?pretty=true" -d '{
  "fields" : ["mapping1.user","mapping2.think"],
"query": {
    "query_string": {
      "query": "kim"
    }
  }
}'

I am getting following results

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 4,
    "max_score" : 0.26516503,
    "hits" : [ {
      "_index" : "index",
      "_type" : "mapping2",
      "_id" : "2",
      "_score" : 0.26516503,
      "fields" : {
        "mapping1.user" : null,
        "mapping2.think" : "2009-11-15T14:12:12"
      }
    }, {
      "_index" : "index",
      "_type" : "mapping2",
      "_id" : "3",
      "_score" : 0.26516503,
      "fields" : {
        "mapping1.user" : null,
        "mapping2.think" : "2009-11-15T14:12:12"
      }
    }, {
      "_index" : "index",
      "_type" : "mapping2",
      "_id" : "1",
      "_score" : 0.18392482,
      "fields" : {
        "mapping1.user" : null,
        "mapping2.think" : "trying out Elastic Search"
      }
    }, {
      "_index" : "index",
      "_type" : "mapping1",
      "_id" : "1",
      "_score" : 0.13137488,
      "fields" : {
        "mapping1.user" : "kim",
        "mapping2.think" : null
      }
    } ]
  }
}

As you can see in above results i've got 4 results out of which only
one i wanted where  "mapping1.user" : "kim" is actually
present.Instead i am receiving all records from mapping2 also where it
doesn't find "user" field and giving me records for  "mapping1.user" :
null.
Solution for that can be to use filters while querying.But my mappings
are containing huge number of columns and also in an index there are
considerably large number of mappings present.
So my questions is now;Whatever be the fields i have mentioned in
"fields" array while querying .query_string should only take those
fields and fire on only those records having fields specified in an
array without filters.
Sorry for such long description.I hope it will be helpful in case.
Thanks;
Arien




On Nov 29, 9:36 pm, Shay Banon <[hidden email]> wrote:

> I did not understand the question properly... . You can specify
> "field_name", but you can also specify "type1.field_name", which will
> automatically wrap the query to be filtered by the type.
>
> On Tue, Nov 29, 2011 at 11:53 AM, arien <[hidden email]> wrote:
> > Hi all;
> > I have some questions when it comes to querying specific fields in
> > multiple mappings provided that all mappings belongs to same index.say
> > "test" is the mapping and "testmap1","testmap2","testmap3" are the
> > mappings of an index.
> > In query i dont want to specify any mapping name just simply wants to
> > put searchable columns there.
> > And the second thing; is it possible to register mapping specific
> > query against particular index,if so please provide some inputs.
>
> > Thanks;
> > Arien
Reply | Threaded
Open this post in threaded view
|

Re: Querying specific fields in multiple mappings

Michael Sick
Arien,

If I read your example/question right, believe you just want to use the TermQuery:

Or TermFilter

For Query vs. Filter, you'll have to think about the memory/perf tradeoffs.

On Wed, Nov 30, 2011 at 6:26 AM, arien <[hidden email]> wrote:
Hi shay,
Sorry for that i didn't frame my questions correctly.
In order the elaborate the scenario lets say i have an index "index"
with two mappings "mapping1","mapping2" and following are the records
in those;
============================================
curl -XPUT 'http://localhost:9200/index/mapping1/1' -d '{
   "user" : "kim",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/2' -d '{
   "user" : "ajay",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/3' -d '{
   "user" : "bhavani",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
============================================
curl -XPUT 'http://localhost:9200/index/mapping2/1' -d '{
   "id" : "kim",
   "think" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/2' -d '{
   "id" : "kim",
   "think" : "2009-11-15T14:12:12"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/3' -d '{
   "id" : "kim",
   "think" : "2009-11-15T14:12:12"
}'
=============================================
and if i Search using query


curl "localhost:9200/index/_search?pretty=true" -d '{
 "fields" : ["mapping1.user","mapping2.think"],
"query": {
   "query_string": {
     "query": "kim"
   }
 }
}'

I am getting following results

{
 "took" : 9,
 "timed_out" : false,
 "_shards" : {
   "total" : 5,
   "successful" : 5,
   "failed" : 0
 },
 "hits" : {
   "total" : 4,
   "max_score" : 0.26516503,
   "hits" : [ {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "2",
     "_score" : 0.26516503,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "2009-11-15T14:12:12"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "3",
     "_score" : 0.26516503,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "2009-11-15T14:12:12"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "1",
     "_score" : 0.18392482,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "trying out Elastic Search"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping1",
     "_id" : "1",
     "_score" : 0.13137488,
     "fields" : {
       "mapping1.user" : "kim",
       "mapping2.think" : null
     }
   } ]
 }
}

As you can see in above results i've got 4 results out of which only
one i wanted where  "mapping1.user" : "kim" is actually
present.Instead i am receiving all records from mapping2 also where it
doesn't find "user" field and giving me records for  "mapping1.user" :
null.
Solution for that can be to use filters while querying.But my mappings
are containing huge number of columns and also in an index there are
considerably large number of mappings present.
So my questions is now;Whatever be the fields i have mentioned in
"fields" array while querying .query_string should only take those
fields and fire on only those records having fields specified in an
array without filters.
Sorry for such long description.I hope it will be helpful in case.
Thanks;
Arien




On Nov 29, 9:36 pm, Shay Banon <[hidden email]> wrote:
> I did not understand the question properly... . You can specify
> "field_name", but you can also specify "type1.field_name", which will
> automatically wrap the query to be filtered by the type.
>
> On Tue, Nov 29, 2011 at 11:53 AM, arien <[hidden email]> wrote:
> > Hi all;
> > I have some questions when it comes to querying specific fields in
> > multiple mappings provided that all mappings belongs to same index.say
> > "test" is the mapping and "testmap1","testmap2","testmap3" are the
> > mappings of an index.
> > In query i dont want to specify any mapping name just simply wants to
> > put searchable columns there.
> > And the second thing; is it possible to register mapping specific
> > query against particular index,if so please provide some inputs.
>
> > Thanks;
> > Arien

Reply | Threaded
Open this post in threaded view
|

Re: Querying specific fields in multiple mappings

kimchy
Administrator
In reply to this post by arien
You get the results because query_string will execute by default against the _all field, unless you specify custom fields to search on. Note, you might be confusing things and think that the top level fields (the ones you provide on the same level as "query") will contraint the query to execute on it, but thats not what they do, they control which fields will be returned from the response. You need to put the fields in the query_string element to control which fields the query_string will execute.

On Wed, Nov 30, 2011 at 1:26 PM, arien <[hidden email]> wrote:
Hi shay,
Sorry for that i didn't frame my questions correctly.
In order the elaborate the scenario lets say i have an index "index"
with two mappings "mapping1","mapping2" and following are the records
in those;
============================================
curl -XPUT 'http://localhost:9200/index/mapping1/1' -d '{
   "user" : "kim",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/2' -d '{
   "user" : "ajay",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping1/3' -d '{
   "user" : "bhavani",
   "post" : "2009-11-15T14:12:12",
   "message" : "trying out Elastic Search"
}'
============================================
curl -XPUT 'http://localhost:9200/index/mapping2/1' -d '{
   "id" : "kim",
   "think" : "trying out Elastic Search"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/2' -d '{
   "id" : "kim",
   "think" : "2009-11-15T14:12:12"
}'
curl -XPUT 'http://localhost:9200/index/mapping2/3' -d '{
   "id" : "kim",
   "think" : "2009-11-15T14:12:12"
}'
=============================================
and if i Search using query


curl "localhost:9200/index/_search?pretty=true" -d '{
 "fields" : ["mapping1.user","mapping2.think"],
"query": {
   "query_string": {
     "query": "kim"
   }
 }
}'

I am getting following results

{
 "took" : 9,
 "timed_out" : false,
 "_shards" : {
   "total" : 5,
   "successful" : 5,
   "failed" : 0
 },
 "hits" : {
   "total" : 4,
   "max_score" : 0.26516503,
   "hits" : [ {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "2",
     "_score" : 0.26516503,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "2009-11-15T14:12:12"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "3",
     "_score" : 0.26516503,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "2009-11-15T14:12:12"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping2",
     "_id" : "1",
     "_score" : 0.18392482,
     "fields" : {
       "mapping1.user" : null,
       "mapping2.think" : "trying out Elastic Search"
     }
   }, {
     "_index" : "index",
     "_type" : "mapping1",
     "_id" : "1",
     "_score" : 0.13137488,
     "fields" : {
       "mapping1.user" : "kim",
       "mapping2.think" : null
     }
   } ]
 }
}

As you can see in above results i've got 4 results out of which only
one i wanted where  "mapping1.user" : "kim" is actually
present.Instead i am receiving all records from mapping2 also where it
doesn't find "user" field and giving me records for  "mapping1.user" :
null.
Solution for that can be to use filters while querying.But my mappings
are containing huge number of columns and also in an index there are
considerably large number of mappings present.
So my questions is now;Whatever be the fields i have mentioned in
"fields" array while querying .query_string should only take those
fields and fire on only those records having fields specified in an
array without filters.
Sorry for such long description.I hope it will be helpful in case.
Thanks;
Arien




On Nov 29, 9:36 pm, Shay Banon <[hidden email]> wrote:
> I did not understand the question properly... . You can specify
> "field_name", but you can also specify "type1.field_name", which will
> automatically wrap the query to be filtered by the type.
>
> On Tue, Nov 29, 2011 at 11:53 AM, arien <[hidden email]> wrote:
> > Hi all;
> > I have some questions when it comes to querying specific fields in
> > multiple mappings provided that all mappings belongs to same index.say
> > "test" is the mapping and "testmap1","testmap2","testmap3" are the
> > mappings of an index.
> > In query i dont want to specify any mapping name just simply wants to
> > put searchable columns there.
> > And the second thing; is it possible to register mapping specific
> > query against particular index,if so please provide some inputs.
>
> > Thanks;
> > Arien

Reply | Threaded
Open this post in threaded view
|

Re: Querying specific fields in multiple mappings

arien
Hi;
Thanks for your inputs,
Accordingly i've framed my query like;

curl -XGET "http://localhost:9200/index/_search?pretty=true" -d '{
  "fields": [
    "mapping1.user",
    "mapping2.think"
  ],
  "query": {
    "query_string": {
      "fields": [
        "mapping1.user",
        "mapping2.think"
      ],
      "query": "kim"
    }
  }
}'


which is returning me wanted results;

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.5906161,
    "hits" : [ {
      "_index" : "index",
      "_type" : "mapping1",
      "_id" : "1",
      "_score" : 0.5906161,
      "fields" : {
        "mapping1.user" : "kim",
        "mapping2.think" : null
      }
    } ]
  }

As you can see I have specified an array of fields in query_string to
control on which fields the query_string will execute.
Is this possible in the case of "textquery" where one can specify same
array to controlling searchable fields without making use of
filters??..

Thanking you all;
arien


On Nov 30, 8:19 pm, Shay Banon <[hidden email]> wrote:

> You get the results because query_string will execute by default against
> the _all field, unless you specify custom fields to search on. Note, you
> might be confusing things and think that the top level fields (the ones you
> provide on the same level as "query") will contraint the query to execute
> on it, but thats not what they do, they control which fields will be
> returned from the response. You need to put the fields in the query_string
> element to control which fields the query_string will execute.
>
> On Wed, Nov 30, 2011 at 1:26 PM, arien <[hidden email]> wrote:
> > Hi shay,
> > Sorry for that i didn't frame my questions correctly.
> > In order the elaborate the scenario lets say i have an index "index"
> > with two mappings "mapping1","mapping2" and following are the records
> > in those;
> > ============================================
> > curl -XPUT 'http://localhost:9200/index/mapping1/1'-d '{
> >    "user" : "kim",
> >    "post" : "2009-11-15T14:12:12",
> >    "message" : "trying out Elastic Search"
> > }'
> > curl -XPUT 'http://localhost:9200/index/mapping1/2'-d '{
> >    "user" : "ajay",
> >    "post" : "2009-11-15T14:12:12",
> >    "message" : "trying out Elastic Search"
> > }'
> > curl -XPUT 'http://localhost:9200/index/mapping1/3'-d '{
> >    "user" : "bhavani",
> >    "post" : "2009-11-15T14:12:12",
> >    "message" : "trying out Elastic Search"
> > }'
> > ============================================
> > curl -XPUT 'http://localhost:9200/index/mapping2/1'-d '{
> >    "id" : "kim",
> >    "think" : "trying out Elastic Search"
> > }'
> > curl -XPUT 'http://localhost:9200/index/mapping2/2'-d '{
> >    "id" : "kim",
> >    "think" : "2009-11-15T14:12:12"
> > }'
> > curl -XPUT 'http://localhost:9200/index/mapping2/3'-d '{
> >    "id" : "kim",
> >    "think" : "2009-11-15T14:12:12"
> > }'
> > =============================================
> > and if i Search using query
>
> > curl "localhost:9200/index/_search?pretty=true" -d '{
> >  "fields" : ["mapping1.user","mapping2.think"],
> > "query": {
> >    "query_string": {
> >      "query": "kim"
> >    }
> >  }
> > }'
>
> > I am getting following results
>
> > {
> >  "took" : 9,
> >  "timed_out" : false,
> >  "_shards" : {
> >    "total" : 5,
> >    "successful" : 5,
> >    "failed" : 0
> >  },
> >  "hits" : {
> >    "total" : 4,
> >    "max_score" : 0.26516503,
> >    "hits" : [ {
> >      "_index" : "index",
> >      "_type" : "mapping2",
> >      "_id" : "2",
> >      "_score" : 0.26516503,
> >      "fields" : {
> >        "mapping1.user" : null,
> >        "mapping2.think" : "2009-11-15T14:12:12"
> >      }
> >    }, {
> >      "_index" : "index",
> >      "_type" : "mapping2",
> >      "_id" : "3",
> >      "_score" : 0.26516503,
> >      "fields" : {
> >        "mapping1.user" : null,
> >        "mapping2.think" : "2009-11-15T14:12:12"
> >      }
> >    }, {
> >      "_index" : "index",
> >      "_type" : "mapping2",
> >      "_id" : "1",
> >      "_score" : 0.18392482,
> >      "fields" : {
> >        "mapping1.user" : null,
> >        "mapping2.think" : "trying out Elastic Search"
> >      }
> >    }, {
> >      "_index" : "index",
> >      "_type" : "mapping1",
> >      "_id" : "1",
> >      "_score" : 0.13137488,
> >      "fields" : {
> >        "mapping1.user" : "kim",
> >        "mapping2.think" : null
> >      }
> >    } ]
> >  }
> > }
>
> > As you can see in above results i've got 4 results out of which only
> > one i wanted where  "mapping1.user" : "kim" is actually
> > present.Instead i am receiving all records from mapping2 also where it
> > doesn't find "user" field and giving me records for  "mapping1.user" :
> > null.
> > Solution for that can be to use filters while querying.But my mappings
> > are containing huge number of columns and also in an index there are
> > considerably large number of mappings present.
> > So my questions is now;Whatever be the fields i have mentioned in
> > "fields" array while querying .query_string should only take those
> > fields and fire on only those records having fields specified in an
> > array without filters.
> > Sorry for such long description.I hope it will be helpful in case.
> > Thanks;
> > Arien
>
> > On Nov 29, 9:36 pm, Shay Banon <[hidden email]> wrote:
> > > I did not understand the question properly... . You can specify
> > > "field_name", but you can also specify "type1.field_name", which will
> > > automatically wrap the query to be filtered by the type.
>
> > > On Tue, Nov 29, 2011 at 11:53 AM, arien <[hidden email]> wrote:
> > > > Hi all;
> > > > I have some questions when it comes to querying specific fields in
> > > > multiple mappings provided that all mappings belongs to same index.say
> > > > "test" is the mapping and "testmap1","testmap2","testmap3" are the
> > > > mappings of an index.
> > > > In query i dont want to specify any mapping name just simply wants to
> > > > put searchable columns there.
> > > > And the second thing; is it possible to register mapping specific
> > > > query against particular index,if so please provide some inputs.
>
> > > > Thanks;
> > > > Arien