Exception with index.store.type: memory

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Exception with index.store.type: memory

Ridvan Gyundogan
Hello,
I run on a single local machine.
This is the elasticsearch.yml:
cluster.name: ridvansg
index :
    store:
        type: memory


When I start I get the followin exception, probably I miss something
in my configuration?

[2011-08-31 14:18:03,212][WARN ][indices.cluster          ] [Jaren]
[giftsplusindex][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[giftsplusindex][3] shard allocated for local recovery (post api),
should exists, but doesn't
        at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:
99)
        at org.elasticsearch.index.gateway.IndexShardGatewayService
$1.run(IndexShardGatewayService.java:179)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

Without the index.store.type: memory  setting the ES 17.4 works fine.

Reply | Threaded
Open this post in threaded view
|

Re: Exception with index.store.type: memory

Ridvan Gyundogan
Hm now I have this exception even after removing the
"index.store.type: memory" setting.

This is how I did it:
1. load 600k small documents - worked fine at the end.
2. Add filter.cache(true) to each filter. - worked fine
3. add index.store.type: memory - exceptions in the elasticsearch log
file.
4. remove index.store.type: memory - still exceptions in the
elasticsearch log file. The ES doesn't start anymore.

On Aug 31, 2:22 pm, Ridvan Gyundogan <[hidden email]> wrote:

> Hello,
> I run on a single local machine.
> This is the elasticsearch.yml:
> cluster.name: ridvansg
> index :
>     store:
>         type: memory
>
> When I start I get the followin exception, probably I miss something
> in my configuration?
>
> [2011-08-31 14:18:03,212][WARN ][indices.cluster          ] [Jaren]
> [giftsplusindex][3] failed to start shard
> org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> [giftsplusindex][3] shard allocated for local recovery (post api),
> should exists, but doesn't
>         at
> org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:
> 99)
>         at org.elasticsearch.index.gateway.IndexShardGatewayService
> $1.run(IndexShardGatewayService.java:179)
>         at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
>
> Without the index.store.type: memory  setting the ES 17.4 works fine.
Reply | Threaded
Open this post in threaded view
|

Re: Exception with index.store.type: memory

Ridvan Gyundogan
I forgot to mention that I restart ES by first kill'ing it. Probably
not the best way?

On Aug 31, 2:33 pm, Ridvan Gyundogan <[hidden email]> wrote:

> Hm now I have this exception even after removing the
> "index.store.type: memory" setting.
>
> This is how I did it:
> 1. load 600k small documents - worked fine at the end.
> 2. Add filter.cache(true) to each filter. - worked fine
> 3. add index.store.type: memory - exceptions in the elasticsearch log
> file.
> 4. remove index.store.type: memory - still exceptions in the
> elasticsearch log file. The ES doesn't start anymore.
>
> On Aug 31, 2:22 pm, Ridvan Gyundogan <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hello,
> > I run on a single local machine.
> > This is the elasticsearch.yml:
> > cluster.name: ridvansg
> > index :
> >     store:
> >         type: memory
>
> > When I start I get the followin exception, probably I miss something
> > in my configuration?
>
> > [2011-08-31 14:18:03,212][WARN ][indices.cluster          ] [Jaren]
> > [giftsplusindex][3] failed to start shard
> > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > [giftsplusindex][3] shard allocated for local recovery (post api),
> > should exists, but doesn't
> >         at
> > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:
> > 99)
> >         at org.elasticsearch.index.gateway.IndexShardGatewayService
> > $1.run(IndexShardGatewayService.java:179)
> >         at java.util.concurrent.ThreadPoolExecutor
> > $Worker.runTask(ThreadPoolExecutor.java:886)
> >         at java.util.concurrent.ThreadPoolExecutor
> > $Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
>
> > Without the index.store.type: memory  setting the ES 17.4 works fine.
Reply | Threaded
Open this post in threaded view
|

Re: Exception with index.store.type: memory

kimchy
Administrator
If you use index.store.type set to memory (which has some problems in the implementation that I have not tracked yet), then using local gateway (the default) will not work. Once you create an index with settings, they are set to those settings, so if you kill elasticsearch and start it again (with local gateway), you will get failures. If you want a clean elasticsearch instance, delete teh data dir.

I suggest you use the file system based one, and if you want extra guice, use the mmapfs option.

On Wed, Aug 31, 2011 at 2:57 PM, Ridvan Gyundogan <[hidden email]> wrote:
I forgot to mention that I restart ES by first kill'ing it. Probably
not the best way?

On Aug 31, 2:33 pm, Ridvan Gyundogan <[hidden email]> wrote:
> Hm now I have this exception even after removing the
> "index.store.type: memory" setting.
>
> This is how I did it:
> 1. load 600k small documents - worked fine at the end.
> 2. Add filter.cache(true) to each filter. - worked fine
> 3. add index.store.type: memory - exceptions in the elasticsearch log
> file.
> 4. remove index.store.type: memory - still exceptions in the
> elasticsearch log file. The ES doesn't start anymore.
>
> On Aug 31, 2:22 pm, Ridvan Gyundogan <[hidden email]> wrote:
>
>
>
>
>
>
>
> > Hello,
> > I run on a single local machine.
> > This is the elasticsearch.yml:
> > cluster.name: ridvansg
> > index :
> >     store:
> >         type: memory
>
> > When I start I get the followin exception, probably I miss something
> > in my configuration?
>
> > [2011-08-31 14:18:03,212][WARN ][indices.cluster          ] [Jaren]
> > [giftsplusindex][3] failed to start shard
> > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > [giftsplusindex][3] shard allocated for local recovery (post api),
> > should exists, but doesn't
> >         at
> > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:
> > 99)
> >         at org.elasticsearch.index.gateway.IndexShardGatewayService
> > $1.run(IndexShardGatewayService.java:179)
> >         at java.util.concurrent.ThreadPoolExecutor
> > $Worker.runTask(ThreadPoolExecutor.java:886)
> >         at java.util.concurrent.ThreadPoolExecutor
> > $Worker.run(ThreadPoolExecutor.java:908)
> >         at java.lang.Thread.run(Thread.java:662)
>
> > Without the index.store.type: memory  setting the ES 17.4 works fine.

Reply | Threaded
Open this post in threaded view
|

Re: Exception with index.store.type: memory

Ridvan Gyundogan
I will try, thanks a lot!

On Aug 31, 3:57 pm, Shay Banon <[hidden email]> wrote:

> If you use index.store.type set to memory (which has some problems in the
> implementation that I have not tracked yet), then using local gateway (the
> default) will not work. Once you create an index with settings, they are set
> to those settings, so if you kill elasticsearch and start it again (with
> local gateway), you will get failures. If you want a clean elasticsearch
> instance, delete teh data dir.
>
> I suggest you use the file system based one, and if you want extra guice,
> use the mmapfs option.
>
> On Wed, Aug 31, 2011 at 2:57 PM, Ridvan Gyundogan <[hidden email]>wrote:
>
>
>
>
>
>
>
> > I forgot to mention that I restart ES by first kill'ing it. Probably
> > not the best way?
>
> > On Aug 31, 2:33 pm, Ridvan Gyundogan <[hidden email]> wrote:
> > > Hm now I have this exception even after removing the
> > > "index.store.type: memory" setting.
>
> > > This is how I did it:
> > > 1. load 600k small documents - worked fine at the end.
> > > 2. Add filter.cache(true) to each filter. - worked fine
> > > 3. add index.store.type: memory - exceptions in the elasticsearch log
> > > file.
> > > 4. remove index.store.type: memory - still exceptions in the
> > > elasticsearch log file. The ES doesn't start anymore.
>
> > > On Aug 31, 2:22 pm, Ridvan Gyundogan <[hidden email]> wrote:
>
> > > > Hello,
> > > > I run on a single local machine.
> > > > This is the elasticsearch.yml:
> > > > cluster.name: ridvansg
> > > > index :
> > > >     store:
> > > >         type: memory
>
> > > > When I start I get the followin exception, probably I miss something
> > > > in my configuration?
>
> > > > [2011-08-31 14:18:03,212][WARN ][indices.cluster          ] [Jaren]
> > > > [giftsplusindex][3] failed to start shard
> > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > > > [giftsplusindex][3] shard allocated for local recovery (post api),
> > > > should exists, but doesn't
> > > >         at
>
> > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:
> > > > 99)
> > > >         at org.elasticsearch.index.gateway.IndexShardGatewayService
> > > > $1.run(IndexShardGatewayService.java:179)
> > > >         at java.util.concurrent.ThreadPoolExecutor
> > > > $Worker.runTask(ThreadPoolExecutor.java:886)
> > > >         at java.util.concurrent.ThreadPoolExecutor
> > > > $Worker.run(ThreadPoolExecutor.java:908)
> > > >         at java.lang.Thread.run(Thread.java:662)
>
> > > > Without the index.store.type: memory  setting the ES 17.4 works fine.