Querystring not being analyzed

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Querystring not being analyzed

Robert Eanes
I'm trying to get a simple case-insensitive search to work by running
the querystring through the "standard" analyzer, but it doesn't seem
to be working for me.  The index seems to be run through the analyzer,
as a querystring that is all lowercase matches a mixed-case record.  A
mixed or uppercase querystring never matches though. I've tried
explicitly setting the analyzer parameter, with no effect.
I'm using 0.4.0, with no changes to the config file (which is empty).

Thanks for any help you can give me - here is an example session
illustrating the problem:

~ $ curl -XPUT http://localhost:9200/twitter/tweet/1 -d \
'
{
    user : "Kimchy",
    postDate : "2009-11-15T14:12:12",
    message : "trying out Elastic Search"
}
'
{"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}

~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
q=user:kimchy"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
{
    user : "Kimchy",
    postDate : "2009-11-15T14:12:12",
    message : "trying out Elastic Search"
}
}]}}

So far, so good.  The lowercase term matched the mixed-case original,
but then:

~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
q=user:KIMCHY"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
0,"hits":[]}}

With explicit analyzer specified:

~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
q=user:KIMCHY&analyzer=standard"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
0,"hits":[]}}


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Querystring not being analyzed

kimchy
Administrator
Hi,

  Yep, this is a bug in how I handle this case in the enhanced query parser in elasticsearch. I have already pushed a fixed. Can you give it a go?

-shay.banon

On Tue, Feb 23, 2010 at 7:18 PM, Robert Eanes <[hidden email]> wrote:
I'm trying to get a simple case-insensitive search to work by running
the querystring through the "standard" analyzer, but it doesn't seem
to be working for me.  The index seems to be run through the analyzer,
as a querystring that is all lowercase matches a mixed-case record.  A
mixed or uppercase querystring never matches though. I've tried
explicitly setting the analyzer parameter, with no effect.
I'm using 0.4.0, with no changes to the config file (which is empty).

Thanks for any help you can give me - here is an example session
illustrating the problem:

~ $ curl -XPUT http://localhost:9200/twitter/tweet/1 -d \
'
{
   user : "Kimchy",
   postDate : "2009-11-15T14:12:12",
   message : "trying out Elastic Search"
}
'
{"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}

~ $ curl -XGET "<a href="http://localhost:9200/twitter/tweet/_search? q=user:kimchy" target="_blank">http://localhost:9200/twitter/tweet/_search?
q=user:kimchy"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
{
   user : "Kimchy",
   postDate : "2009-11-15T14:12:12",
   message : "trying out Elastic Search"
}
}]}}

So far, so good.  The lowercase term matched the mixed-case original,
but then:

~ $ curl -XGET "<a href="http://localhost:9200/twitter/tweet/_search? q=user:KIMCHY" target="_blank">http://localhost:9200/twitter/tweet/_search?
q=user:KIMCHY"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
0,"hits":[]}}

With explicit analyzer specified:

~ $ curl -XGET "<a href="http://localhost:9200/twitter/tweet/_search? q=user:KIMCHY&amp;analyzer=standard" target="_blank">http://localhost:9200/twitter/tweet/_search?
q=user:KIMCHY&analyzer=standard"
{"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
0,"hits":[]}}



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Querystring not being analyzed

Robert Eanes
Thanks for responding so quickly. I'd love to test it out, but I'm
having trouble building the latest version from github.  This is
almost certainly due to my very limited java experience, rather than
an actual problem with the code. For instance I'd never heard of
gradle until today.  In any case, I don't want to bother you with
that, just knowing it's fixed for an upcoming release is great for
me.  Thanks again!

On Feb 23, 3:16 pm, Shay Banon <[hidden email]> wrote:

> Hi,
>
>   Yep, this is a bug in how I handle this case in the enhanced query parser
> in elasticsearch. I have already pushed a fixed. Can you give it a go?
>
> -shay.banon
>
>
>
> On Tue, Feb 23, 2010 at 7:18 PM, Robert Eanes <[hidden email]> wrote:
> > I'm trying to get a simple case-insensitive search to work by running
> > the querystring through the "standard" analyzer, but it doesn't seem
> > to be working for me.  The index seems to be run through the analyzer,
> > as a querystring that is all lowercase matches a mixed-case record.  A
> > mixed or uppercase querystring never matches though. I've tried
> > explicitly setting the analyzer parameter, with no effect.
> > I'm using 0.4.0, with no changes to the config file (which is empty).
>
> > Thanks for any help you can give me - here is an example session
> > illustrating the problem:
>
> > ~ $ curl -XPUThttp://localhost:9200/twitter/tweet/1-d \
> > '
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > '
> > {"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:kimchy"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > }]}}
>
> > So far, so good.  The lowercase term matched the mixed-case original,
> > but then:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}
>
> > With explicit analyzer specified:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY&analyzer=standard"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Querystring not being analyzed

Lukáš Vlček
I think you will not bother anybody when you share your problems. That is what user mail list if for :-)
You can always clone fresh ES code into new directory and run gradlew. This will build fresh ES instance ready for your tests.

Regards,
Lukas

On Wed, Feb 24, 2010 at 4:22 AM, Robert Eanes <[hidden email]> wrote:
Thanks for responding so quickly. I'd love to test it out, but I'm
having trouble building the latest version from github.  This is
almost certainly due to my very limited java experience, rather than
an actual problem with the code. For instance I'd never heard of
gradle until today.  In any case, I don't want to bother you with
that, just knowing it's fixed for an upcoming release is great for
me.  Thanks again!

On Feb 23, 3:16 pm, Shay Banon <[hidden email]> wrote:
> Hi,
>
>   Yep, this is a bug in how I handle this case in the enhanced query parser
> in elasticsearch. I have already pushed a fixed. Can you give it a go?
>
> -shay.banon
>
>
>
> On Tue, Feb 23, 2010 at 7:18 PM, Robert Eanes <[hidden email]> wrote:
> > I'm trying to get a simple case-insensitive search to work by running
> > the querystring through the "standard" analyzer, but it doesn't seem
> > to be working for me.  The index seems to be run through the analyzer,
> > as a querystring that is all lowercase matches a mixed-case record.  A
> > mixed or uppercase querystring never matches though. I've tried
> > explicitly setting the analyzer parameter, with no effect.
> > I'm using 0.4.0, with no changes to the config file (which is empty).
>
> > Thanks for any help you can give me - here is an example session
> > illustrating the problem:
>
> > ~ $ curl -XPUThttp://localhost:9200/twitter/tweet/1-d \
> > '
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > '
> > {"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:kimchy"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > }]}}
>
> > So far, so good.  The lowercase term matched the mixed-case original,
> > but then:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}
>
> > With explicit analyzer specified:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY&analyzer=standard"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Querystring not being analyzed

kimchy
Administrator
Also, make sure you have Java 6 JDK installed (http://java.sun.com/javase/downloads/widget/jdk6.jsp), and for simplicity, set JAVA_HOME to point to the installation, and add $JDK_HOME/bin to your path.

-shay.banon

On Wed, Feb 24, 2010 at 9:09 AM, Lukáš Vlček <[hidden email]> wrote:
I think you will not bother anybody when you share your problems. That is what user mail list if for :-)
You can always clone fresh ES code into new directory and run gradlew. This will build fresh ES instance ready for your tests.

Regards,
Lukas


On Wed, Feb 24, 2010 at 4:22 AM, Robert Eanes <[hidden email]> wrote:
Thanks for responding so quickly. I'd love to test it out, but I'm
having trouble building the latest version from github.  This is
almost certainly due to my very limited java experience, rather than
an actual problem with the code. For instance I'd never heard of
gradle until today.  In any case, I don't want to bother you with
that, just knowing it's fixed for an upcoming release is great for
me.  Thanks again!

On Feb 23, 3:16 pm, Shay Banon <[hidden email]> wrote:
> Hi,
>
>   Yep, this is a bug in how I handle this case in the enhanced query parser
> in elasticsearch. I have already pushed a fixed. Can you give it a go?
>
> -shay.banon
>
>
>
> On Tue, Feb 23, 2010 at 7:18 PM, Robert Eanes <[hidden email]> wrote:
> > I'm trying to get a simple case-insensitive search to work by running
> > the querystring through the "standard" analyzer, but it doesn't seem
> > to be working for me.  The index seems to be run through the analyzer,
> > as a querystring that is all lowercase matches a mixed-case record.  A
> > mixed or uppercase querystring never matches though. I've tried
> > explicitly setting the analyzer parameter, with no effect.
> > I'm using 0.4.0, with no changes to the config file (which is empty).
>
> > Thanks for any help you can give me - here is an example session
> > illustrating the problem:
>
> > ~ $ curl -XPUThttp://localhost:9200/twitter/tweet/1-d \
> > '
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > '
> > {"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:kimchy"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
> > {
> >    user : "Kimchy",
> >    postDate : "2009-11-15T14:12:12",
> >    message : "trying out Elastic Search"
> > }
> > }]}}
>
> > So far, so good.  The lowercase term matched the mixed-case original,
> > but then:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}
>
> > With explicit analyzer specified:
>
> > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > q=user:KIMCHY&analyzer=standard"
> > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > 0,"hits":[]}}


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Querystring not being analyzed

Robert Eanes
Thanks! I was able to successfully build the second time I tried with a clean checkout, and I can confirm that the test in my original email now works as expected.  In case you are interested, here is the log of what I saw during the build, including the failed and successful attempts.  Note that I had to kill it after the stack trace appeared in the first attempt as it was hanging:

~/src $ git clone git://github.com/elasticsearch/elasticsearch.git
Initialized empty Git repository in /Users/reanes/src/elasticsearch/.git/
remote: Counting objects: 4169, done.
remote: Compressing objects: 100% (1909/1909), done.
remote: Total 4169 (delta 2408), reused 3516 (delta 1867)
Receiving objects: 100% (4169/4169), 1.34 MiB | 893 KiB/s, done.
Resolving deltas: 100% (2408/2408), done.
~/src $ cd elasticsearch
~/src/elasticsearch(master) $ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-10M3025)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
~/src/elasticsearch(master) $ ./gradlew build devRelease
:test-testng:compileJava
:elasticsearch:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:test-testng:processResources
:elasticsearch:processResources
:test-testng:classes
:elasticsearch:classes
:test-testng:jar
:elasticsearch:jar
:elasticsearch:uploadDefaultInternal
:benchmark-micro:compileJava
:benchmark-micro:processResources
:benchmark-micro:classes
:benchmark-micro:jar
:test-testng:assemble
:elasticsearch:assemble
:benchmark-micro:assemble
:test-testng:compileTestJava
:test-testng:uploadDefaultInternal
:elasticsearch:compileTestJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:benchmark-micro:compileTestJava
:test-testng:processTestResources
:elasticsearch:processTestResources
:benchmark-micro:processTestResources
:test-testng:testClasses
:elasticsearch:testClasses
:benchmark-micro:testClasses
:test-testng:test
:elasticsearch:test
[ant:testng] ........................................
[ant:testng] ........................................
[ant:testng] ...........Exception in thread "elasticsearch[tp]-pool-10-thread-1" java.lang.AssertionError:
[ant:testng] Expected: "Action [sayHelloException] not found"
[ant:testng]      got: "bad message !!!"
[ant:testng]
[ant:testng]    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:21)
[ant:testng]    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
[ant:testng]    at org.elasticsearch.transport.netty.SimpleNettyTransportTests$4.handleException(SimpleNettyTransportTests.java:133)
[ant:testng]    at org.elasticsearch.transport.PlainTransportFuture.handleException(PlainTransportFuture.java:143)
[ant:testng]    at org.elasticsearch.transport.netty.MessageChannelHandler$2.run(MessageChannelHandler.java:125)
[ant:testng]    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[ant:testng]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[ant:testng]    at java.lang.Thread.run(Thread.java:637)


^C~/src/elasticsearch(master) $ ./gradlew build devRelease
:test-testng:compileJava
:elasticsearch:compileJava
:test-testng:processResources
:elasticsearch:processResources
:test-testng:classes
:elasticsearch:classes
:test-testng:jar
:elasticsearch:jar
:elasticsearch:uploadDefaultInternal
:benchmark-micro:compileJava
:benchmark-micro:processResources
:benchmark-micro:classes
:benchmark-micro:jar
:test-testng:assemble
:elasticsearch:assemble
:benchmark-micro:assemble
:test-testng:compileTestJava
:test-testng:uploadDefaultInternal
:elasticsearch:compileTestJava
:benchmark-micro:compileTestJava
:test-testng:processTestResources
:elasticsearch:processTestResources
:benchmark-micro:processTestResources
:test-testng:testClasses
:elasticsearch:testClasses
:benchmark-micro:testClasses
:test-testng:test
:elasticsearch:test
[ant:testng] ........................................
[ant:testng] ........................................
[ant:testng] ........................................
[ant:testng] .......
:benchmark-micro:test
:test-testng:check
:elasticsearch:check
:benchmark-micro:check
:test-testng:build
:elasticsearch:build
:benchmark-micro:build
:test-integration:compileJava
:test-integration:processResources
:test-integration:classes
:test-integration:jar
[ant:jar] Warning: skipping jar archive /Users/reanes/src/elasticsearch/modules/test/integration/build/libs/elasticsearch-test-integration-0.5.0.jar because no files were included.
[ant:jar] Warning: skipping jar archive /Users/reanes/src/elasticsearch/modules/test/integration/build/libs/elasticsearch-test-integration-0.5.0.jar because no files were included.
:test-integration:assemble
:test-integration:compileTestJava
Note: /Users/reanes/src/elasticsearch/modules/test/integration/src/test/java/org/elasticsearch/test/integration/indexlifecycle/IndexLifecycleActionTests.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:test-integration:processTestResources
:test-integration:testClasses
:test-integration:test
[ant:testng] ........................................
[ant:testng] ...........
:test-integration:check
:test-integration:build
:explodedDist
:zip
:devRelease

BUILD SUCCESSFUL

On Feb 24, 2010, at 3:16 AM, Shay Banon wrote:

> Also, make sure you have Java 6 JDK installed (http://java.sun.com/javase/downloads/widget/jdk6.jsp), and for simplicity, set JAVA_HOME to point to the installation, and add $JDK_HOME/bin to your path.
>
> -shay.banon
>
> On Wed, Feb 24, 2010 at 9:09 AM, Lukáš Vlček <[hidden email]> wrote:
> I think you will not bother anybody when you share your problems. That is what user mail list if for :-)
> You can always clone fresh ES code into new directory and run gradlew. This will build fresh ES instance ready for your tests.
>
> Regards,
> Lukas
>
>
> On Wed, Feb 24, 2010 at 4:22 AM, Robert Eanes <[hidden email]> wrote:
> Thanks for responding so quickly. I'd love to test it out, but I'm
> having trouble building the latest version from github.  This is
> almost certainly due to my very limited java experience, rather than
> an actual problem with the code. For instance I'd never heard of
> gradle until today.  In any case, I don't want to bother you with
> that, just knowing it's fixed for an upcoming release is great for
> me.  Thanks again!
>
> On Feb 23, 3:16 pm, Shay Banon <[hidden email]> wrote:
> > Hi,
> >
> >   Yep, this is a bug in how I handle this case in the enhanced query parser
> > in elasticsearch. I have already pushed a fixed. Can you give it a go?
> >
> > -shay.banon
> >
> >
> >
> > On Tue, Feb 23, 2010 at 7:18 PM, Robert Eanes <[hidden email]> wrote:
> > > I'm trying to get a simple case-insensitive search to work by running
> > > the querystring through the "standard" analyzer, but it doesn't seem
> > > to be working for me.  The index seems to be run through the analyzer,
> > > as a querystring that is all lowercase matches a mixed-case record.  A
> > > mixed or uppercase querystring never matches though. I've tried
> > > explicitly setting the analyzer parameter, with no effect.
> > > I'm using 0.4.0, with no changes to the config file (which is empty).
> >
> > > Thanks for any help you can give me - here is an example session
> > > illustrating the problem:
> >
> > > ~ $ curl -XPUThttp://localhost:9200/twitter/tweet/1-d \
> > > '
> > > {
> > >    user : "Kimchy",
> > >    postDate : "2009-11-15T14:12:12",
> > >    message : "trying out Elastic Search"
> > > }
> > > '
> > > {"ok":true,"_index":"twitter","_type":"tweet","_id":"1"}
> >
> > > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > > q=user:kimchy"
> > > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > > 1,"hits":[{"_index":"twitter","_type":"tweet","_id":"1", "_source" :
> > > {
> > >    user : "Kimchy",
> > >    postDate : "2009-11-15T14:12:12",
> > >    message : "trying out Elastic Search"
> > > }
> > > }]}}
> >
> > > So far, so good.  The lowercase term matched the mixed-case original,
> > > but then:
> >
> > > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > > q=user:KIMCHY"
> > > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > > 0,"hits":[]}}
> >
> > > With explicit analyzer specified:
> >
> > > ~ $ curl -XGET "http://localhost:9200/twitter/tweet/_search?
> > > q=user:KIMCHY&analyzer=standard"
> > > {"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":
> > > 0,"hits":[]}}
>
>

Loading...