Quantcast

Can ES return only the part of an array that match the query

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Can ES return only the part of an array that match the query

Margaret-2
I have documents like this: 

   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 


As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!

--
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can ES return only the part of an array that match the query

dadoonet
You have to look at nested docs: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html
 
HTH
David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :
I have documents like this: 

   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 


As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!
 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can ES return only the part of an array that match the query

Margaret-2
Thanks for reply, David.
 
Let me describe my problem more explicitly.
I tried the example given in Nested Type(http://www.elasticsearch.org/guide/reference/mapping/nested-type.html) and Nested Query:
{
    "query":{
         "nested":{
             "path":"obj1",
             "query":{
                  "bool":{
                      "must":[
                          {
                               "text":{"obj1.name":"blue"}
                           },
                          {
                               "range":{"obj1.count":{"lt":5}}
                           }
                       ]
                   }
              }
          }
      }
}'. 

The query result is:
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915, "_source" :
{
   "obj1":[
     {
       "name":"blue",
       "count":4
      }
      ,{
       "name":"green",
        "count":6
       }
    ]
}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can ES return result like this:
{
   "obj1":[
     {
       "name":"blue",
       "count":4
      }
    ]
}
?

Thanks a lot.


2012/12/14 David Pilato <[hidden email]>
 
HTH
David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :
I have documents like this: 

   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 


As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!
 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 



--
Thanks,

Xiaowen Huang 黄晓雯
Student of School of Software Engineering
Beijing Jiaotong University
Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China
Tel.: +86-152-1058-2730

--
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can ES return only the part of an array that match the query

dadoonet
Can you gist a curl recreation?
It will help to see what is your mapping and how you create your doc.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 déc. 2012 à 04:17, Xiaowen Huang <[hidden email]> a écrit :

Thanks for reply, David.
 
Let me describe my problem more explicitly.
I tried the example given in Nested Type(http://www.elasticsearch.org/guide/reference/mapping/nested-type.html) and Nested Query:
{
    "query":{
         "nested":{
             "path":"obj1",
             "query":{
                  "bool":{
                      "must":[
                          {
                               "text":{"obj1.name":"blue"}
                           },
                          {
                               "range":{"obj1.count":{"lt":5}}
                           }
                       ]
                   }
              }
          }
      }
}'. 

The query result is:
{"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915, "_source" :
{
   "obj1":[
     {
       "name":"blue",
       "count":4
      }
      ,{
       "name":"green",
        "count":6
       }
    ]
}}]}}

But only element {"name":"blue","count":4} in "obj1" matches the query. Can ES return result like this:
{
   "obj1":[
     {
       "name":"blue",
       "count":4
      }
    ]
}
?

Thanks a lot.


2012/12/14 David Pilato <[hidden email]>
 
HTH
David

Le 13 décembre 2012 à 16:42, Margaret <[hidden email]> a écrit :
I have documents like this: 

   ... ..., 
   "id":"abc001", 
   "nominations":[ 
     { 
         "comment":"test", 
         "events":[ 
              { 
                  "name":"aaa", 
                  "year":"2012" 
               },{ 
                  "name":"bbb", 
                  "year":"2011" 
               } 
          ] 
       }, 
       ... ... 
    ] 


As you can see, there are two nested array, 'nominations' and 'events'. 

When I search for docs that match {"term":{"events.name":"aaa"}} and {"term":{"events.year":"2012"}}, the document above will be returned. 

But in practice, array 'events' may contains many elements like "{"name":"bbb","year":"2011"}" that don't match the query and also are useless to users. So my question is, can Elasticsearch return a document with array 'events' that contains element "{"name":"aaa","year":"2012"}" only? If not, is there any other possible ways to solve this problem? Or should I change the structure of the document? 

Thanks a lot!
 

 

--
 
 

 

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--
 
 



--
Thanks,

Xiaowen Huang 黄晓雯
Student of School of Software Engineering
Beijing Jiaotong University
Add.:No.3 Shangyuancun, Haidian District, Beijing, P.R.China
Tel.: +86-152-1058-2730

--
 
 

--
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can ES return only the part of an array that match the query

phill
In reply to this post by Margaret-2
You want less than all the member of the elements in the array.  Oh how
much, I'd like that too.

My understanding is that nested objects can prevent "cross object"
matches (searching for +name:"blue" +count:6 and incorrectly getting a
match), but when you match the parent document with a nested query you
can only get back either
(1) the whole parent (outer) document, or
(2) certain stored fields of the parent
that is all.

I believe at this time, the only thing you can do is ask for documents
that have at least 1 nested or 1 child document that matches.
If you switch to children docs you could:
(1) search for parent documents
(2) search for children documents that match your query of children, but
have the right parents.

Then combine the results.

If the criteria for matching children is not for scoring the relevancy
of the parent, you can put the matching of children in a filter and make
sure that result is cached, so even though you are going back again and
repeating the same possibly complex filter, the filtered children is
already done and it need only be further filtered by the requested
parent IDs.

Because of filter caching two requests can be very fast.

-Paul

On 12/13/2012 7:17 PM, Xiaowen Huang wrote:

> The query result is:
> {"took":0,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.724915,"hits":[{"_index":"test","_type":"es","_id":"GrmFSJo-RSetmjevjloA-Q","_score":1.724915,
> "_source" :
> {
>    "obj1":[
>      {
>        "name":"blue",
>        "count":4
>       }
>       ,{
>        "name":"green",
>         "count":6
>        }
>     ]
> }}]}}
>
> But only element {"name":"blue","count":4} in "obj1" matches the
> query. Can ES return result like this:
> {
>    "obj1":[
>      {
>        "name":"blue",
>        "count":4
>       }
>     ]
> }
> ?
>
> Thanks a lot.
>

--


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can ES return only the part of an array that match the query

senthild4u@gmail.com
This post has NOT been accepted by the mailing list yet.
Hi,

I also have the similar issue. nested type still returns me whole whole array instead of matching array. do we ahve a solution for this problem ?
Loading...