SearchHit.fields() returns null even though fields are there

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

SearchHit.fields() returns null even though fields are there

John Chang
Just upgraded from 0.9 to 0.11.0 and I find that org.elasticsearch.search.SearchHit.fields() is returning null even when I know the requested fields are there.  When I do the query from the url through curl (including the requested fields), they show up fine in the returned json.  

How I query:
response = indexClient.search(Requests.searchRequest(getIndexName()).types(documentTypes).searchType(SearchType.QUERY_THEN_FETCH)
                       .source(SearchSourceBuilder.searchSource().query(QueryBuilders.customScoreQuery(QueryBuilders.queryString(query)).script("doc['" + sortField + "'].value")).fields(fields).from(0).size(maxHitsPerIndexQuery).explain(true))


How I get the results:
        List<Map<String, List<Object>>> results = new ArrayList<Map<String, List<Object>>>();
        for(SearchHit hit : response.hits().hits()) {

            Map<String, List<Object>> fieldMap = new HashMap<String, List<Object>>();

            for (Map.Entry<String, SearchHitField> field : hit.fields().entrySet()) {  //NPE here
                fieldMap.put(field.getKey(), field.getValue().getValues());
            }

            results.add(fieldMap);
        }

Note this is not a problem when I start local nodes in my unit tests; I only see this problem when I connect to a true stand-alone node started from the command line.


Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: SearchHit.fields() returns null even though fields are there

kimchy
Administrator
Sounds like a serialization problem... . Is there a chance that you can create a simple test case and send it to me?

On Sat, Oct 9, 2010 at 3:00 AM, John Chang <[hidden email]> wrote:

Just upgraded from 0.9 to 0.11.0 and I find that
org.elasticsearch.search.SearchHit.fields() is returning null even when I
know the requested fields are there.  When I do the query from the url
through curl (including the requested fields), they show up fine in the
returned json.

How I query:
response =
indexClient.search(Requests.searchRequest(getIndexName()).types(documentTypes).searchType(SearchType.QUERY_THEN_FETCH)

.source(SearchSourceBuilder.searchSource().query(QueryBuilders.customScoreQuery(QueryBuilders.queryString(query)).script("doc['"
+ sortField +
"'].value")).fields(fields).from(0).size(maxHitsPerIndexQuery).explain(true))


How I get the results:
       List<Map<String, List>> results = new ArrayList<Map<String,
List>>();
       for(SearchHit hit : response.hits().hits()) {

           Map<String, List> fieldMap = new HashMap<String, List>();

           for (Map.Entry<String, SearchHitField> field :
hit.fields().entrySet()) {  //NPE here
               fieldMap.put(field.getKey(), field.getValue().getValues());
           }

           results.add(fieldMap);
       }

Note this is not a problem when I start local nodes in my unit tests; I only
see this problem when I connect to a true stand-alone node started from the
command line.


Thanks.
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1668529.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: SearchHit.fields() returns null even though fields are there

John Chang
Thanks for the offer, but I think I may have figured it out.

I simplified my client setup to the below snippet.  I took out the index conifg stuff (as you advised in a different thread) because I was setting client(true).  After doing this, the null fields problem went away.  Does that make sense; would that explain it?

BTW - the problem was erratic.  It generally gave me back null fields, but sometimes they where non-null but empty, and sometimes they were non-null and properly populated.

At any rate, since dropping the index config and simplifying the client setup to what you see below, it seems to work, at least for now.


    private void initClient() {
        //elasticSearchHostsList should only ever be null or empty in unit test invocation
        if (elasticSearchHostsList != null && !elasticSearchHostsList.trim().equals("")) {
            NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
            nb.settings().put("discovery.zen.ping.unicast.hosts", elasticSearchHostsList);

            //cluster name must change as per environment to avoid s3 path clashes -- only in junit test invocation should it be left to default
            if (elasticSearchClusterName != null && !elasticSearchClusterName.trim().equals("")) {
                nb.settings().put("cluster.name", elasticSearchClusterName);
            }

            this.indexClient = nb.node().client();

            logger.info("Successfully initialized client with Elastic Search hosts " + elasticSearchHostsList);
        }

    }
Reply | Threaded
Open this post in threaded view
|

Re: SearchHit.fields() returns null even though fields are there

kimchy
Administrator
Strange, this should not have fixed the problem ... . Is there a chance, if you have time, for a recreation of this so I can delve into why this was happening?

On Mon, Oct 11, 2010 at 9:13 AM, John Chang <[hidden email]> wrote:

Thanks for the offer, but I think I may have figured it out.

I simplified my client setup to the below snippet.  I took out the index
conifg stuff (as you advised in a different thread) because I was setting
client(true).  After doing this, the null fields problem went away.  Does
that make sense; would that explain it?

BTW - the problem was erratic.  It generally gave me back null fields, but
sometimes they where non-null but empty, and sometimes they were non-null
and properly populated.

At any rate, since dropping the index config and simplifying the client
setup to what you see below, it seems to work, at least for now.


   private void initClient() {
       //elasticSearchHostsList should only ever be null or empty in unit
test invocation
       if (elasticSearchHostsList != null &&
!elasticSearchHostsList.trim().equals("")) {
           NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
           nb.settings().put("discovery.zen.ping.unicast.hosts",
elasticSearchHostsList);

           //cluster name must change as per environment to avoid s3 path
clashes -- only in junit test invocation should it be left to default
           if (elasticSearchClusterName != null &&
!elasticSearchClusterName.trim().equals("")) {
               nb.settings().put("cluster.name", elasticSearchClusterName);
           }

           this.indexClient = nb.node().client();

           logger.info("Successfully initialized client with Elastic Search
hosts " + elasticSearchHostsList);
       }

   }
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1678640.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: SearchHit.fields() returns null even though fields are there

John Chang
I'll try, but as I said, it is inconsistent.  When I tried to write a little stand-alone test case to repro the problem, it worked fine.  Of course, I can't ship you my whole project for numerous reasons.  I'll try again to get a stand-alone repro test case to send you, but it may be elusive.  THanks.
Reply | Threaded
Open this post in threaded view
|

Re: SearchHit.fields() returns null even though fields are there

kimchy
Administrator
Hey,

  If you do manage to recreate it, it would be great. I will try and run some tests on my end to see if I can reproduce this as well. If its a bug, I would love to nail it.

-shay.banon

On Tue, Oct 12, 2010 at 1:24 AM, John Chang <[hidden email]> wrote:

I'll try, but as I said, it is inconsistent.  When I tried to write a little
stand-alone test case to repro the problem, it worked fine.  Of course, I
can't ship you my whole project for numerous reasons.  I'll try again to get
a stand-alone repro test case to send you, but it may be elusive.  THanks.
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/SearchHit-fields-returns-null-even-though-fields-are-there-tp1668529p1684188.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.