Quantcast

index mapping via template file

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

index mapping via template file

Ulli
Hi,

I'm trying to create an index mapping via template file as descirbed here: http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html
So I created the following template file in config/templates/template_1.json:

{
    "template_1" : {
        "template" : "*",
        "mappings" : {
            "date" : {
                "index" : "not_analyzed",
                "type" : "date",
                "format" : "dd/MMM/yyyy:HH:mm:ss Z"
            }
        }
    }
}

with no effect:

curl -XGET localhost:9200/_template/template_1
{}

When I put the mapping via curl request:

curl -XPUT localhost:9200/_template/template_1 -d '
{
    "template" : "*",
    "mappings" : {
        "date" : {
            "index" : "not_analyzed",
            "type" : "date",
            "format" : "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
}
'

it works:

curl -XGET localhost:9200/_template/template_1?pretty=true
{
  "template_1" : {
    "template" : "*",
    "order" : 0,
    "settings" : { },
    "mappings" : {
      "date" : {
        "index" : "not_analyzed",
        "format" : "dd/MMM/yyyy:HH:mm:ss Z",
        "type" : "date"
      }
    }
  }
}

So, what do I have to do to make it working via template file?

Thanks in advance
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
The configuration of path.conf was wrong but now I see an exception in the log:

org.elasticsearch.index.mapper.MapperParsingException: mapping [date]
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:262)
        at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:208)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Trying to parse an object but has a different type [date] for [date]
        at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:206)
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:161)
        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:364)
        at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:186)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:259)


Am Donnerstag, 14. Februar 2013 16:57:56 UTC+1 schrieb Ulli:
Hi,

I'm trying to create an index mapping via template file as descirbed here: http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html
So I created the following template file in config/templates/template_1.json:

{
    "template_1" : {
        "template" : "*",
        "mappings" : {
            "date" : {
                "index" : "not_analyzed",
                "type" : "date",
                "format" : "dd/MMM/yyyy:HH:mm:ss Z"
            }
        }
    }
}

with no effect:

curl -XGET localhost:9200/_template/template_1
{}

When I put the mapping via curl request:

curl -XPUT localhost:9200/_template/template_1 -d '
{
    "template" : "*",
    "mappings" : {
        "date" : {
            "index" : "not_analyzed",
            "type" : "date",
            "format" : "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
}
'

it works:

curl -XGET localhost:9200/_template/template_1?pretty=true
{
  "template_1" : {
    "template" : "*",
    "order" : 0,
    "settings" : { },
    "mappings" : {
      "date" : {
        "index" : "not_analyzed",
        "format" : "dd/MMM/yyyy:HH:mm:ss Z",
        "type" : "date"
      }
    }
  }
}

So, what do I have to do to make it working via template file?

Thanks in advance
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Does anybody knows what exactly 'Trying to parse an object but has a different type [date] for [date]' mean?

So far I've found out that the second '[date]' is the date I configured in the mapping and try to index as a field. If that means that my input field has not the type 'date' is there a way to find out what elasticsearch 'thinks' the type of my field is?

Am Donnerstag, 14. Februar 2013 17:14:32 UTC+1 schrieb Ulli:
The configuration of path.conf was wrong but now I see an exception in the log:

org.elasticsearch.index.mapper.MapperParsingException: mapping [date]
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:262)
        at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:208)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Trying to parse an object but has a different type [date] for [date]
        at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:206)
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:161)
        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:364)
        at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:186)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:259)


Am Donnerstag, 14. Februar 2013 16:57:56 UTC+1 schrieb Ulli:
Hi,

I'm trying to create an index mapping via template file as descirbed here: http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html
So I created the following template file in config/templates/template_1.json:

{
    "template_1" : {
        "template" : "*",
        "mappings" : {
            "date" : {
                "index" : "not_analyzed",
                "type" : "date",
                "format" : "dd/MMM/yyyy:HH:mm:ss Z"
            }
        }
    }
}

with no effect:

curl -XGET localhost:9200/_template/template_1
{}

When I put the mapping via curl request:

curl -XPUT localhost:9200/_template/template_1 -d '
{
    "template" : "*",
    "mappings" : {
        "date" : {
            "index" : "not_analyzed",
            "type" : "date",
            "format" : "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
}
'

it works:

curl -XGET localhost:9200/_template/template_1?pretty=true
{
  "template_1" : {
    "template" : "*",
    "order" : 0,
    "settings" : { },
    "mappings" : {
      "date" : {
        "index" : "not_analyzed",
        "format" : "dd/MMM/yyyy:HH:mm:ss Z",
        "type" : "date"
      }
    }
  }
}

So, what do I have to do to make it working via template file?

Thanks in advance
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Radu Gheorghe-2
Hello Ulli,

Elasticsearch expects a mapping type there where you put "date", not a field name.

So like when you store a document in http://localhost:9200/test_index/test_type/test_document_ID, "test_type" is your mapping type.

So your template has to be something like this:

[...]
"mappings": {
    "type1": {
        "properties": {
            "date": {
                "type": "date"
            }
        }
    }
}

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Fri, Feb 15, 2013 at 12:52 PM, Ulli <[hidden email]> wrote:
Does anybody knows what exactly 'Trying to parse an object but has a different type [date] for [date]' mean?

So far I've found out that the second '[date]' is the date I configured in the mapping and try to index as a field. If that means that my input field has not the type 'date' is there a way to find out what elasticsearch 'thinks' the type of my field is?

Am Donnerstag, 14. Februar 2013 17:14:32 UTC+1 schrieb Ulli:
The configuration of path.conf was wrong but now I see an exception in the log:

org.elasticsearch.index.mapper.MapperParsingException: mapping [date]
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:262)
        at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:208)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Trying to parse an object but has a different type [date] for [date]
        at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:206)
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:161)
        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:364)
        at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:186)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:259)


Am Donnerstag, 14. Februar 2013 16:57:56 UTC+1 schrieb Ulli:
Hi,

I'm trying to create an index mapping via template file as descirbed here: http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html
So I created the following template file in config/templates/template_1.json:

{
    "template_1" : {
        "template" : "*",
        "mappings" : {
            "date" : {
                "index" : "not_analyzed",
                "type" : "date",
                "format" : "dd/MMM/yyyy:HH:mm:ss Z"
            }
        }
    }
}

with no effect:

curl -XGET localhost:9200/_template/template_1
{}

When I put the mapping via curl request:

curl -XPUT localhost:9200/_template/template_1 -d '
{
    "template" : "*",
    "mappings" : {
        "date" : {
            "index" : "not_analyzed",
            "type" : "date",
            "format" : "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
}
'

it works:

curl -XGET localhost:9200/_template/template_1?pretty=true
{
  "template_1" : {
    "template" : "*",
    "order" : 0,
    "settings" : { },
    "mappings" : {
      "date" : {
        "index" : "not_analyzed",
        "format" : "dd/MMM/yyyy:HH:mm:ss Z",
        "type" : "date"
      }
    }
  }
}

So, what do I have to do to make it working via template file?

Thanks in advance
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Thank you, Radu. It looks better know. I don't see any exceptions in the log any more. But a range search on the date field doesn't work.

This is what my mapping looks like now:

{
    "template_1" : {
        "template" : "*",
        "access-log" : {
            "properties" : {
                "date" : {
                    "index" : "not_analyzed",
                    "type" : "date",
                    "format" : "dd/MMM/yyyy:HH:mm:ss Z"
                }
            }
        }
    }
}

I indexed one document with field that has the date "12/Sep/2012:11:24:02 +0200"
But the following range search has 0 hits:

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{
  "query":{
    "bool":{
      "must":[
        {
          "range":{
            "accessdate":{
              "from":"12/Sep/2012:11:24:00 +0200",
              "to":"12/Sep/2012:11:24:05 +0200"
            }
          }
        }
      ]
    }
  }
}'

But this search finds the document:

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{
  "query":{
    "bool":{
      "must":[
        {
          "range":{
            "accessdate":{
              "from":"10/Sep/2012:23:59:59 +0200",
              "to":"11/Sep/2012:00:00:00 +0200"
            }
          }
        }
      ]
    }
  }
}'

It seems that search is successful when the dates have different days regardless of what is indexed. I tried to find out the 'real' date in the index is:

curl'http://localhost:9200/_all0/_search?pretty=true' -d '{
    "query" : {
        "match_all" : { }
    },       
    "script_fields": {
        "terms" : {
            "script": "doc[field].values",
            "params": {                          
                "field": "accessdate"
            }
        }
    }   
}'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "logstash-2013.02.18",
      "_type" : "access-log",
      "_id" : "apV2kN7pT8yFmmFnT_W_Mg",
      "_score" : 1.0,
      "fields" : {
        "terms" : [ "02", "0200", "11", "12", "2012", "24", "sep" ]
      }
    } ]
  }
}

I don't know if this is something I can rely on. The date in the index should be stored as milliseconds since epoche, right? Does this output means my date is not stored as a date?

Am Samstag, 16. Februar 2013 14:24:01 UTC+1 schrieb Radu Gheorghe:
Hello Ulli,

Elasticsearch expects a mapping type there where you put "date", not a field name.

So like when you store a document in http://localhost:9200/test_index/test_type/test_document_ID, "test_type" is your mapping type.

So your template has to be something like this:

[...]
"mappings": {
    "type1": {
        "properties": {
            "date": {
                "type": "date"
            }
        }
    }
}

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Fri, Feb 15, 2013 at 12:52 PM, Ulli <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="DBociNH_EWEJ">ulli....@...> wrote:
Does anybody knows what exactly 'Trying to parse an object but has a different type [date] for [date]' mean?

So far I've found out that the second '[date]' is the date I configured in the mapping and try to index as a field. If that means that my input field has not the type 'date' is there a way to find out what elasticsearch 'thinks' the type of my field is?

Am Donnerstag, 14. Februar 2013 17:14:32 UTC+1 schrieb Ulli:
The configuration of path.conf was wrong but now I see an exception in the log:

org.elasticsearch.index.mapper.MapperParsingException: mapping [date]
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:262)
        at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:208)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Trying to parse an object but has a different type [date] for [date]
        at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parse(ObjectMapper.java:206)
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:161)
        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:364)
        at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:186)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:259)


Am Donnerstag, 14. Februar 2013 16:57:56 UTC+1 schrieb Ulli:
Hi,

I'm trying to create an index mapping via template file as descirbed here: http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html
So I created the following template file in config/templates/template_1.json:

{
    "template_1" : {
        "template" : "*",
        "mappings" : {
            "date" : {
                "index" : "not_analyzed",
                "type" : "date",
                "format" : "dd/MMM/yyyy:HH:mm:ss Z"
            }
        }
    }
}

with no effect:

curl -XGET localhost:9200/_template/template_1
{}

When I put the mapping via curl request:

curl -XPUT localhost:9200/_template/template_1 -d '
{
    "template" : "*",
    "mappings" : {
        "date" : {
            "index" : "not_analyzed",
            "type" : "date",
            "format" : "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
}
'

it works:

curl -XGET localhost:9200/_template/template_1?pretty=true
{
  "template_1" : {
    "template" : "*",
    "order" : 0,
    "settings" : { },
    "mappings" : {
      "date" : {
        "index" : "not_analyzed",
        "format" : "dd/MMM/yyyy:HH:mm:ss Z",
        "type" : "date"
      }
    }
  }
}

So, what do I have to do to make it working via template file?

Thanks in advance
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="DBociNH_EWEJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Obviously the mapping is not enough:

curl -XGET 'http://localhost:9200/_all/_mapping?pretty=true'
{
  "logstash-2013.02.18" : {
    "access-log" : {
      "properties" : {
        "@fields" : {
          "dynamic" : "true",
          "properties" : {
            "ZONE" : {
              "type" : "string"
            },
            "accessdate" : {
              "type" : "string"
            },
            ...

Do I have to tell elasticsearch that my date field 'accessdate' is of type date? How?


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Radu Gheorghe-2
Hello Ulli,

Yes, you have to tell Elasticsearch that your accessdate field is date. That's what you define in your mapping.

Please note that you also need to point to the specific field in your query. So '@fields.accessdate' instead of just 'accessdate'.

I've put together a gist with a sample mapping and sample query that should work. I didn't get to test them, so if there are mistakes please let me know.

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Tue, Feb 19, 2013 at 10:53 AM, Ulli <[hidden email]> wrote:
Obviously the mapping is not enough:

curl -XGET 'http://localhost:9200/_all/_mapping?pretty=true'
{
  "logstash-2013.02.18" : {
    "access-log" : {
      "properties" : {
        "@fields" : {
          "dynamic" : "true",
          "properties" : {
            "ZONE" : {
              "type" : "string"
            },
            "accessdate" : {
              "type" : "string"
            },
            ...

Do I have to tell elasticsearch that my date field 'accessdate' is of type date? How?



--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Am Dienstag, 19. Februar 2013 14:24:56 UTC+1 schrieb Radu Gheorghe:
I've put together a gist with a sample mapping and sample query that should work.

Thank you. Where do I find it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Radu Gheorghe-2
Jeez,  forgot to put the link. Sorry. Here you go:
https://gist.github.com/radu-gheorghe/4985795


On Tue, Feb 19, 2013 at 3:44 PM, Ulli <[hidden email]> wrote:
Am Dienstag, 19. Februar 2013 14:24:56 UTC+1 schrieb Radu Gheorghe:
I've put together a gist with a sample mapping and sample query that should work.

Thank you. Where do I find it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
So, when I put your sample into my template file as described in my original posting I should see 'accessdate' as type date when using curl -XGET 'http://localhost:9200/_all/_mapping?pretty=true'?
It's still a string:

curl -XGET 'http://localhost:9200/_all/_mapping?pretty=true'
{
  "logstash-2013.02.19" : {
    "access-log" : {
      "properties" : {
        "@fields" : {
          "dynamic" : "true",
          "properties" : {
            "ZONE" : {
              "type" : "string"
            },
            "accessdate" : {
              "type" : "string"
            },
            ...

Am Dienstag, 19. Februar 2013 15:58:29 UTC+1 schrieb Radu Gheorghe:
Jeez,  forgot to put the link. Sorry. Here you go:
https://gist.github.com/radu-gheorghe/4985795


On Tue, Feb 19, 2013 at 3:44 PM, Ulli <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="zHsJSGoxRzMJ">ulli....@...> wrote:
Am Dienstag, 19. Februar 2013 14:24:56 UTC+1 schrieb Radu Gheorghe:
I've put together a gist with a sample mapping and sample query that should work.

Thank you. Where do I find it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="zHsJSGoxRzMJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Radu Gheorghe-2
Hello Ulli,

When you change your mapping, you need to remove your data (indices) and reindex that data. The template will only be applied when a new index (that matches the template name) is created.

You can "extend" the mapping by using the Put Mapping API:
http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html

But even then, you can change a field's type without reindexing.

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Tue, Feb 19, 2013 at 5:58 PM, Ulli <[hidden email]> wrote:
So, when I put your sample into my template file as described in my original posting I should see 'accessdate' as type date when using curl -XGET 'http://localhost:9200/_all/_mapping?pretty=true'?
It's still a string:  "logstash-2013.02.19" : {

    "access-log" : {
      "properties" : {
        "@fields" : {
          "dynamic" : "true",
          "properties" : {
            "ZONE" : {
              "type" : "string"
            },
            "accessdate" : {
              "type" : "string"
            },
            ...

Am Dienstag, 19. Februar 2013 15:58:29 UTC+1 schrieb Radu Gheorghe:
Jeez,  forgot to put the link. Sorry. Here you go:
https://gist.github.com/radu-gheorghe/4985795


On Tue, Feb 19, 2013 at 3:44 PM, Ulli <[hidden email]> wrote:
Am Dienstag, 19. Februar 2013 14:24:56 UTC+1 schrieb Radu Gheorghe:
I've put together a gist with a sample mapping and sample query that should work.

Thank you. Where do I find it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Am Dienstag, 19. Februar 2013 19:09:41 UTC+1 schrieb Radu Gheorghe:
When you change your mapping, you need to remove your data (indices) and reindex that data.


I know, I did it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
Is there a way to verify that the template file is actually used by elasticsearch?

Am Mittwoch, 20. Februar 2013 09:43:45 UTC+1 schrieb Ulli:
Am Dienstag, 19. Februar 2013 19:09:41 UTC+1 schrieb Radu Gheorghe:
When you change your mapping, you need to remove your data (indices) and reindex that data.


I know, I did it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Radu Gheorghe-2
Hello Ulli,

The only way I'm aware of is to test it: create an index that matches the template and get the mapping for the type you're interested in.

Best regards,
Radu
--
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene

On Wed, Feb 20, 2013 at 11:55 AM, Ulli <[hidden email]> wrote:
Is there a way to verify that the template file is actually used by elasticsearch?

Am Mittwoch, 20. Februar 2013 09:43:45 UTC+1 schrieb Ulli:
Am Dienstag, 19. Februar 2013 19:09:41 UTC+1 schrieb Radu Gheorghe:
When you change your mapping, you need to remove your data (indices) and reindex that data.


I know, I did it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: index mapping via template file

Ulli
This helped my to get it work with 'curl XPUT'. This it what it looks now:

Code hier eingeben...curl -XPUT http://localhost:9200/_template/logstash_per_index -d '{
   
"template" : "logstash*",
   
"mappings" : {
       
"_default_" : {
           
"_all" : {"enabled" : false},
           
"properties" : {
             
"@fields" : {
                   
"type" : "object",
                   
"dynamic": true,
                   
"path": "full",
                   
"properties" : {
                       
"accessdate" : {
                           
"index" : "not_analyzed",
                           
"type" : "date",
                           
"format" : "dd/MMM/yyyy:HH:mm:ss Z"
                       
}
                   
}
             
},
             
"@message": { "type": "string", "index": "analyzed" },
             
"@source": { "type": "string", "index": "not_analyzed" },
             
"@source_host": { "type": "string", "index": "not_analyzed" },
             
"@source_path": { "type": "string", "index": "not_analyzed" },
             
"@tags": { "type": "string", "index": "not_analyzed" },
             
"@timestamp": { "type": "date", "index": "not_analyzed" },
             
"@type": { "type": "string", "index": "not_analyzed" }    
           
}  
       
}
   
}
}
'

But I'm still trying to get it work with a template file. There are many different and contradictory statements about where to place it and how to name it, e.g.:

  • config/default-mapping.json
  • config/mappings/_default/<type>.json
  • config/mappings/<template-name>.json
  • ...
So, what is the right way? If there is <type> used in a file name is it the index type or the data type? Does the root element in the config file has to be the type (which one again)?

Thanks in advance to anyone that gets my out of this confusion.
Ulli

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Loading...