Can ES return only the part of an array that match the query

[Margaret-2]

I have documents like this: 
{ 
   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 
} 

As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!

--
 
 

[dadoonet]

You have to look at nested docs: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

 

HTH

David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :

I have documents like this: 
{ 
   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 
} 

As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!

 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 

[Margaret-2]

Thanks for reply, David.

 

Let me describe my problem more explicitly.

I tried the example given in Nested Type(http://www.elasticsearch.org/guide/reference/mapping/nested-type.html) and Nested Query:

$ curl -XGET 'http://localhost:9200/test/es/_search' -d '

{

    "query":{

         "nested":{

             "path":"obj1",

             "query":{

                  "bool":{

                      "must":[

                          {

                               "text":{"obj1.name":"blue"}

                           },

                          {

                               "range":{"obj1.count":{"lt":5}}

                           }

                       ]

                   }

              }

          }

      }

}'. 

The query result is:

{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915, "_source" :

{

   "obj1":[

     {

       "name":"blue",

       "count":4

      }

      ,{

       "name":"green",

        "count":6

       }

    ]

}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can ES return result like this:

{

   "obj1":[

     {

       "name":"blue",

       "count":4

      }

    ]

}

?

Thanks a lot.

2012/12/14 David Pilato <[hidden email]>

You have to look at nested docs: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

 

HTH

David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :

I have documents like this: 
{ 
   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 
} 

As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!

 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 

--

Thanks,

Xiaowen Huang | 黄晓雯

Student of School of Software Engineering

Beijing Jiaotong University

Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China

Tel.: [phone]

Email: [hidden email]

--
 
 

[dadoonet]

Can you gist a curl recreation?

It will help to see what is your mapping and how you create your doc.

--

David ;-)

Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 déc. 2012 à 04:17, Xiaowen Huang <[hidden email]> a écrit :

Thanks for reply, David.

 

Let me describe my problem more explicitly.

I tried the example given in Nested Type(http://www.elasticsearch.org/guide/reference/mapping/nested-type.html) and Nested Query:

$ curl -XGET 'http://localhost:9200/test/es/_search' -d '

{

    "query":{

         "nested":{

             "path":"obj1",

             "query":{

                  "bool":{

                      "must":[

                          {

                               "text":{"obj1.name":"blue"}

                           },

                          {

                               "range":{"obj1.count":{"lt":5}}

                           }

                       ]

                   }

              }

          }

      }

}'. 

The query result is:

{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915, "_source" :

{

   "obj1":[

     {

       "name":"blue",

       "count":4

      }

      ,{

       "name":"green",

        "count":6

       }

    ]

}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can ES return result like this:

{

   "obj1":[

     {

       "name":"blue",

       "count":4

      }

    ]

}

?

Thanks a lot.

2012/12/14 David Pilato <[hidden email]>

You have to look at nested docs: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

 

HTH

David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :

I have documents like this: 
{ 
   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 
} 

As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!

 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 

--

Thanks,

Xiaowen Huang | 黄晓雯

Student of School of Software Engineering

Beijing Jiaotong University

Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China

Tel.: [phone]

Email: [hidden email]

--
 
 

--
 
 

[phill]

You want less than all the member of the elements in the array.  Oh how
much, I'd like that too.

My understanding is that nested objects can prevent "cross object"
matches (searching for +name:"blue" +count:6 and incorrectly getting a
match), but when you match the parent document with a nested query you
can only get back either
(1) the whole parent (outer) document, or
(2) certain stored fields of the parent
that is all.

I believe at this time, the only thing you can do is ask for documents
that have at least 1 nested or 1 child document that matches.
If you switch to children docs you could:
(1) search for parent documents
(2) search for children documents that match your query of children, but
have the right parents.

Then combine the results.

If the criteria for matching children is not for scoring the relevancy
of the parent, you can put the matching of children in a filter and make
sure that result is cached, so even though you are going back again and
repeating the same possibly complex filter, the filtered children is
already done and it need only be further filtered by the requested
parent IDs.

Because of filter caching two requests can be very fast.

-Paul

On 12/13/2012 7:17 PM, Xiaowen Huang wrote:

> The query result is:
> {"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915,
> "_source" :
> {
>    "obj1":[
>      {
>        "name":"blue",
>        "count":4
>       }
>       ,{
>        "name":"green",
>         "count":6
>        }
>     ]
> }}]}}
>
> But only element {"name":"blue","count":4} in "obj1" matches the
> query. Can ES return result like this:
> {
>    "obj1":[
>      {
>        "name":"blue",
>        "count":4
>       }
>     ]
> }
> ?
>
> Thanks a lot.
>

--

Hi,

I also have the similar issue. nested type still returns me whole whole array instead of matching array. do we ahve a solution for this problem ?