Percolate existing document using query from another index

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Percolate existing document using query from another index

Poomrat Boonyawong
Hi, I am trying to follow advice in http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html#_dedicated_percolator_index but percolating on existing document.
This for some reason doesn't work for me in ES 1.1.0.

The same percolator works (match document) if it is in the same index as target document.

Here's my test document
# curl -XGET http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw

{
"_index" : "logstash-2014.04.18",
"_type" : "apache_access",
"_id" : "HAtwipHtSnOmr8ZUn4Q4tw",
"_version" : 1,
"found" : true,
"_source" : {
"@version" : "1",
"@timestamp" : "2014-04-18T08:50:48.461+00:00",
"type" : "apache_access",
"tags" : ["input_nxlog_json", "logstash-ingestion-node-PVCM-EMSINGS01", "logstash-index-node-PVCM-EMSINDX01", "_grokparsefailure"],
"host" : "pvcm-emshead01",
"message" : "10.42.78.81 - - [18/Apr/2014:08:50:47 +0000] GET /monitor/events/css/bootstrap-responsive.min.css HTTP/1.1 304 -",
"itic-lsg-code" : "HEAD",
    ...
}
}
 
If I add percolator like this

curl -XPUT http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/.percolator/head_lsg_in_logstash_index -d '{ "query" : { "bool" : { "must" : [{ "match" : { "host" : "pvcm-emshead01" } }, { "match" : { "itic-lsg-code" : "HEAD" } } ] } } }'

It match this document fine
curl -XGET http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw/_percolate

{
"took" : 2,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"total" : 1,
"matches" : [{
"_index" : "logstash-2014.04.18",
"_id" : "head_lsg_in_logstash_index"
}
]
}

But if I add the same percolator to dedicated index
curl -XPUT http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/.percolator/head_lsg -d '{
"query" : {
"bool" : {
"must" : [{
"match" : {
"host" : "pvcm-emshead01"
}
}, {
"match" : {
"itic-lsg-code" : "HEAD"
}
}
]
}
}
}'

I can't find any combination that use percolator in dedicated index against existing document. Here's combination I've tried to no avail
  • Set percolate_index and percolate_type and call percolate against existing document - get match from percolator in document's index instead
curl -XGET http://pvcm-emsdata01.amers1b.ciscloud:9200/logstash-2014.04.18/apache_access/HAtwipHtSnOmr8ZUn4Q4tw/_percolate -d'{
   "percolate_index" : "percolator",
   "percolate_type" : "apache_access"
}'

{"took":2,"_shards":{"total":2,"successful":2,"failed":0},"total":1,"matches":[{"_index":"logstash-2014.04.18","_id":"head_lsg_in_logstash_index"}]}
 
  • Try calling percolate from dedicated index - error at run time
curl -XPOST http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/.percolator/_percolate -d'{
    "id": "HAtwipHtSnOmr8ZUn4Q4tw",
    "percolate_index" : "logstash-2014.04.18",
    "percolate_type" : "apache_access"
}'

{"took":1,"_shards":{"total":2,"successful":0,"failed":2,"failures":[{"index":"percolator","shard":0,"reason":"BroadcastShardOperationFailedException[[percolator][0] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate]; "},{"index":"percolator","shard":1,"reason":"BroadcastShardOperationFailedException[[percolator][1] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate]; "}]},"total":0,"matches":[]}

Although if I try that percolator against new document (using "doc"), it matches just fine. 
curl -XGET http://pvcm-emsdata01.amers1b.ciscloud:9200/percolator/apache_access/_percolate -d'{
    "doc" : {
   "type": "apache_access",
     "tags":       [
        "input_nxlog_json",
        "logstash-ingestion-node-PVCM-EMSINGS01",
        "logstash-index-node-PVCM-EMSINDX01",
        "_grokparsefailure"
     ],
     "host": "pvcm-emshead01",
     "message": "10.42.78.81 - - [21/Apr/2014:09:25:47 +0000] GET /monitor/events/css/bootstrap.light.min.css HTTP/1.1 304 -",
     "itic-lsg-code": "HEAD"
    }
}'

 {"took":2,"_shards":{"total":2,"successful":2,"failed":0},"total":2,"matches":[{"_index":"percolator","_id":"head_lsg"},{"_index":"percolator","_id":"apache_access_get_default"}]}

Can someone show me how this use case can be done or is it even supported?

Many thanks in advance

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3eabdc40-9eb2-4f57-9f01-30c94fe9a3e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Percolate existing document using query from another index

Binh Ly-2
If you have a doc in an index A and you want to percolate it against queries in index B, you'll need to specify the percolate_index (and potentially percolate_type) url param:

curl -XPOST "http://localhost:9200/a/doc/1/_percolate?pretty&percolate_index=b"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5526868d-5b1f-46a2-b488-a22357f3ed10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Percolate existing document using query from another index

Poomrat Boonyawong
Now that is just what I need.
Thanks a ton for this pointer.

On Tuesday, April 22, 2014 5:06:00 AM UTC+7, Binh Ly wrote:
If you have a doc in an index A and you want to percolate it against queries in index B, you'll need to specify the percolate_index (and potentially percolate_type) url param:

curl -XPOST "<a href="http://localhost:9200/a/doc/1/_percolate?pretty&amp;percolate_index=b" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Flocalhost%3A9200%2Fa%2Fdoc%2F1%2F_percolate%3Fpretty%26percolate_index%3Db\46sa\75D\46sntz\0751\46usg\75AFQjCNGLEmlUCIer3DBmegQ1DnYKfo1NFQ';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Flocalhost%3A9200%2Fa%2Fdoc%2F1%2F_percolate%3Fpretty%26percolate_index%3Db\46sa\75D\46sntz\0751\46usg\75AFQjCNGLEmlUCIer3DBmegQ1DnYKfo1NFQ';return true;">http://localhost:9200/a/doc/1/_percolate?pretty&percolate_index=b"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/663672a2-8bf0-4fe6-a73b-740004c3ccb3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.