Mapping Clarification

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Mapping Clarification

Matt-2
I'd like to create a new index on some time interval (day or week)
where all indexes have the same format, in order to grow the total
number of shards over time and allow for the total number of servers
in the cluster to grow over time. This happens nicely and
automatically if I just add a date stamp to my index name and continue
indexing records as I get them. Is there a way to define a mapping
that will be inherited by indexes that are created automatically by
Elastic Search?
Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

kimchy
Administrator
There isn't an option to do that currently, the ability for an index created to automatically get a set of mappings applied to it. Open an issue for it, should be simple to implement :)

-shay.banon

On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]> wrote:
I'd like to create a new index on some time interval (day or week)
where all indexes have the same format, in order to grow the total
number of shards over time and allow for the total number of servers
in the cluster to grow over time. This happens nicely and
automatically if I just add a date stamp to my index name and continue
indexing records as I get them. Is there a way to define a mapping
that will be inherited by indexes that are created automatically by
Elastic Search?

Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

Matt-2
Great. My thought was that something like:

curl -XPUT http://localhost:9200/_all/object_name/_mapping -d pay_load

would apply the mapping to current indexes, as well as future indexes.
Does that make sense? Should I phrase the issue as an enhancement to
multi-index mapping, or a totally new feature?

On Mar 24, 3:00 pm, Shay Banon <[hidden email]> wrote:

> There isn't an option to do that currently, the ability for an index created
> to automatically get a set of mappings applied to it. Open an issue for it,
> should be simple to implement :)
>
> -shay.banon
>
>
>
> On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]> wrote:
> > I'd like to create a new index on some time interval (day or week)
> > where all indexes have the same format, in order to grow the total
> > number of shards over time and allow for the total number of servers
> > in the cluster to grow over time. This happens nicely and
> > automatically if I just add a date stamp to my index name and continue
> > indexing records as I get them. Is there a way to define a mapping
> > that will be inherited by indexes that are created automatically by
> > Elastic Search?
Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

kimchy
Administrator
Actually, thats a great idea, I did not think about exposing it as an API, but allowing to specify a list of mappings to load when started. Yours is much better. Just create an issue as a new feature.

I need to think about the syntax though, since you can already do _all, which applies it to the current indices (and not future ones, as its not implemented yet :) ).

-shay.banon

On Wed, Mar 24, 2010 at 9:14 PM, Matt <[hidden email]> wrote:
Great. My thought was that something like:

curl -XPUT http://localhost:9200/_all/object_name/_mapping -d pay_load

would apply the mapping to current indexes, as well as future indexes.
Does that make sense? Should I phrase the issue as an enhancement to
multi-index mapping, or a totally new feature?

On Mar 24, 3:00 pm, Shay Banon <[hidden email]> wrote:
> There isn't an option to do that currently, the ability for an index created
> to automatically get a set of mappings applied to it. Open an issue for it,
> should be simple to implement :)
>
> -shay.banon
>
>
>
> On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]> wrote:
> > I'd like to create a new index on some time interval (day or week)
> > where all indexes have the same format, in order to grow the total
> > number of shards over time and allow for the total number of servers
> > in the cluster to grow over time. This happens nicely and
> > automatically if I just add a date stamp to my index name and continue
> > indexing records as I get them. Is there a way to define a mapping
> > that will be inherited by indexes that are created automatically by
> > Elastic Search?

Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

Matt-2
Your way would be nice too. If I could store my mappings in a config
file and have them apply to all future indexes, I wouldn't have to
clutter my application code at all; the mappings would just be picked
up by the cluster at start-up time. As an API, it's sort of an extra
manual task when you boot a cluster to define the mappings, but it
would allow for adding a new type of object to a growing set of
indices. Maybe I'll create a ticket for each and let you decide what's
a better idea.

On Mar 24, 4:03 pm, Shay Banon <[hidden email]> wrote:

> Actually, thats a great idea, I did not think about exposing it as an API,
> but allowing to specify a list of mappings to load when started. Yours is
> much better. Just create an issue as a new feature.
>
> I need to think about the syntax though, since you can already do _all,
> which applies it to the current indices (and not future ones, as its not
> implemented yet :) ).
>
> -shay.banon
>
>
>
> On Wed, Mar 24, 2010 at 9:14 PM, Matt <[hidden email]> wrote:
> > Great. My thought was that something like:
>
> > curl -XPUThttp://localhost:9200/_all/object_name/_mapping-d pay_load
>
> > would apply the mapping to current indexes, as well as future indexes.
> > Does that make sense? Should I phrase the issue as an enhancement to
> > multi-index mapping, or a totally new feature?
>
> > On Mar 24, 3:00 pm, Shay Banon <[hidden email]> wrote:
> > > There isn't an option to do that currently, the ability for an index
> > created
> > > to automatically get a set of mappings applied to it. Open an issue for
> > it,
> > > should be simple to implement :)
>
> > > -shay.banon
>
> > > On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]> wrote:
> > > > I'd like to create a new index on some time interval (day or week)
> > > > where all indexes have the same format, in order to grow the total
> > > > number of shards over time and allow for the total number of servers
> > > > in the cluster to grow over time. This happens nicely and
> > > > automatically if I just add a date stamp to my index name and continue
> > > > indexing records as I get them. Is there a way to define a mapping
> > > > that will be inherited by indexes that are created automatically by
> > > > Elastic Search?
Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

kimchy
Administrator
cool, I will try and work on it soon.

-shay.banon

On Wed, Mar 24, 2010 at 10:20 PM, Matt <[hidden email]> wrote:
Your way would be nice too. If I could store my mappings in a config
file and have them apply to all future indexes, I wouldn't have to
clutter my application code at all; the mappings would just be picked
up by the cluster at start-up time. As an API, it's sort of an extra
manual task when you boot a cluster to define the mappings, but it
would allow for adding a new type of object to a growing set of
indices. Maybe I'll create a ticket for each and let you decide what's
a better idea.

On Mar 24, 4:03 pm, Shay Banon <[hidden email]> wrote:
> Actually, thats a great idea, I did not think about exposing it as an API,
> but allowing to specify a list of mappings to load when started. Yours is
> much better. Just create an issue as a new feature.
>
> I need to think about the syntax though, since you can already do _all,
> which applies it to the current indices (and not future ones, as its not
> implemented yet :) ).
>
> -shay.banon
>
>
>
> On Wed, Mar 24, 2010 at 9:14 PM, Matt <[hidden email]> wrote:
> > Great. My thought was that something like:
>
> > curl -XPUThttp://localhost:9200/_all/object_name/_mapping-d pay_load
>
> > would apply the mapping to current indexes, as well as future indexes.
> > Does that make sense? Should I phrase the issue as an enhancement to
> > multi-index mapping, or a totally new feature?
>
> > On Mar 24, 3:00 pm, Shay Banon <[hidden email]> wrote:
> > > There isn't an option to do that currently, the ability for an index
> > created
> > > to automatically get a set of mappings applied to it. Open an issue for
> > it,
> > > should be simple to implement :)
>
> > > -shay.banon
>
> > > On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]> wrote:
> > > > I'd like to create a new index on some time interval (day or week)
> > > > where all indexes have the same format, in order to grow the total
> > > > number of shards over time and allow for the total number of servers
> > > > in the cluster to grow over time. This happens nicely and
> > > > automatically if I just add a date stamp to my index name and continue
> > > > indexing records as I get them. Is there a way to define a mapping
> > > > that will be inherited by indexes that are created automatically by
> > > > Elastic Search?

Reply | Threaded
Open this post in threaded view
|

Re: Mapping Clarification

Matt-2
Awesome. Thanks for being so quick to respond.

On Mar 24, 5:28 pm, Shay Banon <[hidden email]> wrote:

> cool, I will try and work on it soon.
>
> -shay.banon
>
>
>
> On Wed, Mar 24, 2010 at 10:20 PM, Matt <[hidden email]> wrote:
> > Your way would be nice too. If I could store my mappings in a config
> > file and have them apply to all future indexes, I wouldn't have to
> > clutter my application code at all; the mappings would just be picked
> > up by the cluster at start-up time. As an API, it's sort of an extra
> > manual task when you boot a cluster to define the mappings, but it
> > would allow for adding a new type of object to a growing set of
> > indices. Maybe I'll create a ticket for each and let you decide what's
> > a better idea.
>
> > On Mar 24, 4:03 pm, Shay Banon <[hidden email]> wrote:
> > > Actually, thats a great idea, I did not think about exposing it as an
> > API,
> > > but allowing to specify a list of mappings to load when started. Yours is
> > > much better. Just create an issue as a new feature.
>
> > > I need to think about the syntax though, since you can already do _all,
> > > which applies it to the current indices (and not future ones, as its not
> > > implemented yet :) ).
>
> > > -shay.banon
>
> > > On Wed, Mar 24, 2010 at 9:14 PM, Matt <[hidden email]> wrote:
> > > > Great. My thought was that something like:
>
> > > > curl -XPUThttp://localhost:9200/_all/object_name/_mapping-dpay_load
>
> > > > would apply the mapping to current indexes, as well as future indexes.
> > > > Does that make sense? Should I phrase the issue as an enhancement to
> > > > multi-index mapping, or a totally new feature?
>
> > > > On Mar 24, 3:00 pm, Shay Banon <[hidden email]> wrote:
> > > > > There isn't an option to do that currently, the ability for an index
> > > > created
> > > > > to automatically get a set of mappings applied to it. Open an issue
> > for
> > > > it,
> > > > > should be simple to implement :)
>
> > > > > -shay.banon
>
> > > > > On Wed, Mar 24, 2010 at 8:52 PM, Matt <[hidden email]>
> > wrote:
> > > > > > I'd like to create a new index on some time interval (day or week)
> > > > > > where all indexes have the same format, in order to grow the total
> > > > > > number of shards over time and allow for the total number of
> > servers
> > > > > > in the cluster to grow over time. This happens nicely and
> > > > > > automatically if I just add a date stamp to my index name and
> > continue
> > > > > > indexing records as I get them. Is there a way to define a mapping
> > > > > > that will be inherited by indexes that are created automatically by
> > > > > > Elastic Search?