Optimize in Elastic search

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Optimize in Elastic search

senthil prabhu
Hi All,

        I have few questions about Optimize operation in Elastic search

1. What is the optimization actually doing and How long does it take?
2. What situations or requirements will cause Elastic Search to start
an optimization?
3. Are there options to when an optimization will start? Or is it
always automatic at the discretion of ES.
4. What are the results if ES is not optimized?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimize in Elastic search

kimchy
Administrator

On Friday, May 6, 2011 at 8:30 AM, senthil prabhu wrote:

Hi All,

I have few questions about Optimize operation in Elastic search

1. What is the optimization actually doing and How long does it take?
Internally, each Lucene index is consisted of several internal segments. On going, Lucene maintains keeping the number of segments at 10 (configurable) by continuously merging segments.

In general, less segments means faster searches and less memory usage, but, more merging required if indexing is done.
2. What situations or requirements will cause Elastic Search to start
an optimization?
Lucene actually maintains that based on the configurable number of segments per index (shard). 
3. Are there options to when an optimization will start? Or is it
always automatic at the discretion of ES.
Automatic. 
4. What are the results if ES is not optimized?
It is always keeps on being optimized. The optimization API allows to explicitly optimize the index. 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimize in Elastic search

senthil prabhu
thank you shay.... from your answer, i am getting some doubts.....

1. How to identify that there optimization process is happening in
ES....?

2. While optimizing index, is it possible to access an index and
process on it...?

3. How long does the optimization process will take....?

On May 6, 8:22 pm, Shay Banon <[hidden email]> wrote:

> On Friday, May 6, 2011 at 8:30 AM, senthil prabhu wrote:
>
> Hi All,
>
> >  I have few questions about Optimize operation in Elastic search
>
> > 1. What is the optimization actually doing and How long does it take?
>
> Internally, each Lucene index is consisted of several internal segments. On going, Lucene maintains keeping the number of segments at 10 (configurable) by continuously merging segments.
>
> In general, less segments means faster searches and less memory usage, but, more merging required if indexing is done.> 2. What situations or requirements will cause Elastic Search to start
> > an optimization?
>
> Lucene actually maintains that based on the configurable number of segments per index (shard).> 3. Are there options to when an optimization will start? Or is it
> > always automatic at the discretion of ES.
> Automatic.
> > 4. What are the results if ES is not optimized?
>
> It is always keeps on being optimized. The optimization API allows to explicitly optimize the index.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimize in Elastic search

kimchy
Administrator

On Saturday, May 7, 2011 at 7:48 AM, senthil prabhu wrote:

thank you shay.... from your answer, i am getting some doubts.....

1. How to identify that there optimization process is happening in
ES....?
It happens all the time. You can get merge stats for it. 

2. While optimizing index, is it possible to access an index and
process on it...?
It happens all the time. You can index and search while merging is going. 

3. How long does the optimization process will take....?
Depends on how big the merges are. 

On May 6, 8:22 pm, Shay Banon <shay.ba...@elasticsearch.com> wrote:
On Friday, May 6, 2011 at 8:30 AM, senthil prabhu wrote:

Hi All,

 I have few questions about Optimize operation in Elastic search

1. What is the optimization actually doing and How long does it take?

Internally, each Lucene index is consisted of several internal segments. On going, Lucene maintains keeping the number of segments at 10 (configurable) by continuously merging segments.

In general, less segments means faster searches and less memory usage, but, more merging required if indexing is done.> 2. What situations or requirements will cause Elastic Search to start
an optimization?

Lucene actually maintains that based on the configurable number of segments per index (shard).> 3. Are there options to when an optimization will start? Or is it
always automatic at the discretion of ES.
Automatic.
4. What are the results if ES is not optimized?

It is always keeps on being optimized. The optimization API allows to explicitly optimize the index.

Loading...