synonym behavior

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

synonym behavior

paul
My synonym.txt contains the below entry

wfu, sso, united search

Now i indexed a field
{
   "name":"sso"
}

when i search as "united search" OR "united" OR "search" it gives sso as the result , below is a sample query
{
  "query": {
    "match": {
      "name": "search"
    }
  }
}

Now i update synonym.txt file with "Help Me" and restart the node . so the synonym.txt content will become
wfu, sso, united search, Help Me
Now if i search for "Help" OR "Me" no results but if i search for "Help Me" will get sso as result. why??
I should get result on "Help" OR "Me" also Since its a query time analysis , the below query returns no results
{
  "query": {
    "match": {
      "name": "Help"
    }
  }
}


My mapping, settings is as below

{
  "settings": {
    "analysis": {
      "analyzer": {
        "synonym": {
          "tokenizer": "whitespace",
          "filter": [
            "standard",
            "lowercase",
            "syns_filter"
          ]
        }
      },
      "filter": {
        "syns_filter": {
          "synonyms_path": "synonyms/synonym_country.txt",
          "type": "synonym"
        }
      }
    }
  },
  "mappings": {
    "hometwo": {
      "properties": {
        "name": {
          "type": "string",
          "analyzer": "synonym"
        }
      }
    }
  }
}

ElasticSearch version 0.90.0

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56e7f43a-7e7b-4a96-a107-9dade8c513f1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: synonym behavior

Ivan Brusic
Note that your filter chain has the synonym filter after the lowercase filter. The terms that are passed through the analyzer will be lowercased, but no match will be found in the synonym map if the terms are also not lowercased. Not sure if this scenario is your problem, but try setting ignore_case to true in synonym filter or use only lowercased terms.

Cheers,

Ivan


On Wed, Dec 11, 2013 at 1:39 AM, paul <[hidden email]> wrote:
My synonym.txt contains the below entry

wfu, sso, united search

Now i indexed a field
{
   "name":"sso"
}

when i search as "united search" OR "united" OR "search" it gives sso as the result , below is a sample query
{
  "query": {
    "match": {
      "name": "search"
    }
  }
}

Now i update synonym.txt file with "Help Me" and restart the node . so the synonym.txt content will become
wfu, sso, united search, Help Me
Now if i search for "Help" OR "Me" no results but if i search for "Help Me" will get sso as result. why??
I should get result on "Help" OR "Me" also Since its a query time analysis , the below query returns no results
{
  "query": {
    "match": {
      "name": "Help"
    }
  }
}


My mapping, settings is as below

{
  "settings": {
    "analysis": {
      "analyzer": {
        "synonym": {
          "tokenizer": "whitespace",
          "filter": [
            "standard",
            "lowercase",
            "syns_filter"
          ]
        }
      },
      "filter": {
        "syns_filter": {
          "synonyms_path": "synonyms/synonym_country.txt",
          "type": "synonym"
        }
      }
    }
  },
  "mappings": {
    "hometwo": {
      "properties": {
        "name": {
          "type": "string",
          "analyzer": "synonym"
        }
      }
    }
  }
}

ElasticSearch version 0.90.0

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56e7f43a-7e7b-4a96-a107-9dade8c513f1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD8OzVG70ymQrUz8cQWrsDhoTVqQq-MOkVrEcQv%2BJ_7ag%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: synonym behavior

paul
Thank you for the reply , I even tried lower case but its still the same behavior.

- Paul


On Wed, Dec 11, 2013 at 11:29 PM, Ivan Brusic <[hidden email]> wrote:
Note that your filter chain has the synonym filter after the lowercase filter. The terms that are passed through the analyzer will be lowercased, but no match will be found in the synonym map if the terms are also not lowercased. Not sure if this scenario is your problem, but try setting ignore_case to true in synonym filter or use only lowercased terms.

Cheers,

Ivan


On Wed, Dec 11, 2013 at 1:39 AM, paul <[hidden email]> wrote:
My synonym.txt contains the below entry

wfu, sso, united search

Now i indexed a field
{
   "name":"sso"
}

when i search as "united search" OR "united" OR "search" it gives sso as the result , below is a sample query
{
  "query": {
    "match": {
      "name": "search"
    }
  }
}

Now i update synonym.txt file with "Help Me" and restart the node . so the synonym.txt content will become
wfu, sso, united search, Help Me
Now if i search for "Help" OR "Me" no results but if i search for "Help Me" will get sso as result. why??
I should get result on "Help" OR "Me" also Since its a query time analysis , the below query returns no results
{
  "query": {
    "match": {
      "name": "Help"
    }
  }
}


My mapping, settings is as below

{
  "settings": {
    "analysis": {
      "analyzer": {
        "synonym": {
          "tokenizer": "whitespace",
          "filter": [
            "standard",
            "lowercase",
            "syns_filter"
          ]
        }
      },
      "filter": {
        "syns_filter": {
          "synonyms_path": "synonyms/synonym_country.txt",
          "type": "synonym"
        }
      }
    }
  },
  "mappings": {
    "hometwo": {
      "properties": {
        "name": {
          "type": "string",
          "analyzer": "synonym"
        }
      }
    }
  }
}

ElasticSearch version 0.90.0

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/HF771t7ocf8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD8OzVG70ymQrUz8cQWrsDhoTVqQq-MOkVrEcQv%2BJ_7ag%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO066G2WORTnynRLSRUihZgqbsXLFopr9-3JJHtKfeQU-zX-Ug%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.