synonym not working for some entry

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

synonym not working for some entry

paul
The synonym entries marked in yellow does not work When i search mit it does not show any results but it works for other entries

ES version 0.90.0
Java Version "1.7.0_45"

my synonym .txt

russia, ussr, russian federation
fui, Florida International University
rate, rating ,rated
mit, mass
jkt, JKTyres

My setting and mapping file

{
   "settings":{
      "analysis":{
         "analyzer":{
            "synonym":{
               "tokenizer":"whitespace",
               "filter":[
                  "standard",
                  "lowercase",
                  "syns_filter"
               ]
            }
         },
         "filter":{
            "syns_filter":{
               "synonyms_path":"synonyms/synonym_country.txt",
               "type":"synonym"
            }
         }
      }
   },
   "mappings":{
      "home":{
         "properties":{
            "name":{
               "type":"string",
               "analyzer":"synonym"
            }
         }
      }
   }
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: synonym not working for some entry

Jun Ohtani
Hi paul,

What kind of your document and query, you fail to search?

I think that “fui” and “jkt” synonym entries does not work, because “Florida International University” and “JKTyres” include UpperCase character.
Your filter chain has the synonym filter after the lowercase filter.
Your document data indexed all lowercase terms by your analyzer, but your query (ex: “jkt") expand “jkt” OR “JKTyres” by synonym filter.

However, I don’t know why “mlt” entry does not work, because “mit” entry is all lowercase character.

Does it make sense?

Regards,

------------
Jun Ohtani
[hidden email]
blog : http://blog.johtani.info
twitter : http://twitter.com/johtani




2013/12/14 22:24、paul  のメール:

> The synonym entries marked in yellow does not work When i search mit it does not show any results but it works for other entries
>
> ES version 0.90.0
> Java Version "1.7.0_45"
>
> my synonym .txt
>
> russia, ussr, russian federation
> fui, Florida International University
> rate, rating ,rated
> mit, mass
> jkt, JKTyres
>
> My setting and mapping file
>
> {
>    "settings":{
>       "analysis":{
>          "analyzer":{
>             "synonym":{
>                "tokenizer":"whitespace",
>                "filter":[
>                   "standard",
>                   "lowercase",
>                   "syns_filter"
>                ]
>             }
>          },
>          "filter":{
>             "syns_filter":{
>                "synonyms_path":"synonyms/synonym_country.txt",
>                "type":"synonym"
>             }
>          }
>       }
>    },
>    "mappings":{
>       "home":{
>          "properties":{
>             "name":{
>                "type":"string",
>                "analyzer":"synonym"
>             }
>          }
>       }
>    }
> }
>
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


signature.asc (507 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: synonym not working for some entry

paul
thank you for the response i will add "ignore_case":true and test


On Sat, Dec 14, 2013 at 7:33 PM, Jun Ohtani <[hidden email]> wrote:
Hi paul,

What kind of your document and query, you fail to search?

I think that “fui” and “jkt” synonym entries does not work, because “Florida International University” and “JKTyres” include UpperCase character.
Your filter chain has the synonym filter after the lowercase filter.
Your document data indexed all lowercase terms by your analyzer, but your query (ex: “jkt") expand “jkt” OR “JKTyres” by synonym filter.

However, I don’t know why “mlt” entry does not work, because “mit” entry is all lowercase character.

Does it make sense?

Regards,

------------
Jun Ohtani
[hidden email]
blog : http://blog.johtani.info
twitter : http://twitter.com/johtani




2013/12/14 22:24、paul  のメール:

> The synonym entries marked in yellow does not work When i search mit it does not show any results but it works for other entries
>
> ES version 0.90.0
> Java Version "1.7.0_45"
>
> my synonym .txt
>
> russia, ussr, russian federation
> fui, Florida International University
> rate, rating ,rated
> mit, mass
> jkt, JKTyres
>
> My setting and mapping file
>
> {
>    "settings":{
>       "analysis":{
>          "analyzer":{
>             "synonym":{
>                "tokenizer":"whitespace",
>                "filter":[
>                   "standard",
>                   "lowercase",
>                   "syns_filter"
>                ]
>             }
>          },
>          "filter":{
>             "syns_filter":{
>                "synonyms_path":"synonyms/synonym_country.txt",
>                "type":"synonym"
>             }
>          }
>       }
>    },
>    "mappings":{
>       "home":{
>          "properties":{
>             "name":{
>                "type":"string",
>                "analyzer":"synonym"
>             }
>          }
>       }
>    }
> }
>
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO066G3FXzCZOutt0iMVo-3H9bnos_pHWP%3DLNhi86WtNe0L%3DkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: synonym not working for some entry

paul
In reply to this post by Jun Ohtani
Hi Jun,
Adding "ignore_case":true worked and for the "mit" there was some special character between words a double space or something mit, mass marked in red when i changed it to no space it worked. is there a specific format for synonym file and what should be the character encoding.


On Saturday, 14 December 2013 19:33:21 UTC+5:30, Jun Ohtani wrote:
Hi paul,

What kind of your document and query, you fail to search?

I think that “fui” and “jkt” synonym entries does not work, because “Florida International University” and “JKTyres” include UpperCase character.
Your filter chain has the synonym filter after the lowercase filter.
Your document data indexed all lowercase terms by your analyzer, but your query (ex: “jkt") expand “jkt” OR “JKTyres” by synonym filter.

However, I don’t know why “mlt” entry does not work, because “mit” entry is all lowercase character.

Does it make sense?

Regards,

------------
Jun Ohtani
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="3TBOk9ns4OkJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">joh...@...
blog : <a href="http://blog.johtani.info" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fblog.johtani.info\46sa\75D\46sntz\0751\46usg\75AFQjCNHuXIdb5WOWj1cqwY0UY_X1E6c1kw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fblog.johtani.info\46sa\75D\46sntz\0751\46usg\75AFQjCNHuXIdb5WOWj1cqwY0UY_X1E6c1kw';return true;">http://blog.johtani.info
twitter : <a href="http://twitter.com/johtani" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Ftwitter.com%2Fjohtani\46sa\75D\46sntz\0751\46usg\75AFQjCNHmhGn5Q8Cg4YDKLur7K2vxOcQjGA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Ftwitter.com%2Fjohtani\46sa\75D\46sntz\0751\46usg\75AFQjCNHmhGn5Q8Cg4YDKLur7K2vxOcQjGA';return true;">http://twitter.com/johtani




2013/12/14 22:24、paul  のメール:

> The synonym entries marked in yellow does not work When i search mit it does not show any results but it works for other entries
>
> ES version 0.90.0
> Java Version "1.7.0_45"
>
> my synonym .txt
>
> russia, ussr, russian federation
> fui, Florida International University
> rate, rating ,rated
> mit, mass
> jkt, JKTyres
>
> My setting and mapping file
>
> {
>    "settings":{
>       "analysis":{
>          "analyzer":{
>             "synonym":{
>                "tokenizer":"whitespace",
>                "filter":[
>                   "standard",
>                   "lowercase",
>                   "syns_filter"
>                ]
>             }
>          },
>          "filter":{
>             "syns_filter":{
>                "synonyms_path":"synonyms/synonym_country.txt",
>                "type":"synonym"
>             }
>          }
>       }
>    },
>    "mappings":{
>       "home":{
>          "properties":{
>             "name":{
>                "type":"string",
>                "analyzer":"synonym"
>             }
>          }
>       }
>    }
> }
>
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="3TBOk9ns4OkJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">elasticsearc...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com';return true;">https://groups.google.com/d/msgid/elasticsearch/05d67cdd-0c9c-4e2a-a9d4-97c48f912f7d%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/groups/opt_out" target="_blank" onmousedown="this.href='https://groups.google.com/groups/opt_out';return true;" onclick="this.href='https://groups.google.com/groups/opt_out';return true;">https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e3f99085-b34d-4636-8f37-46e85e2f2f3c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.