Problem with realtime GET

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with realtime GET

Pavel Penchev
Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a project where I need to guarantee that once a document is index, all subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a document and then immediately reads it. The test works ok on a single node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here - https://gist.github.com/1221648. That class is all you need to run the test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel
Reply | Threaded
Open this post in threaded view
|

Re: Problem with realtime GET

kimchy
Administrator
Hey, thanks for the recreation. Managed to recreate it as well here, opened an issue and fixed it in 0.17 and master branch: https://github.com/elasticsearch/elasticsearch/issues/1344.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev <[hidden email]> wrote:
Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a project where I need to guarantee that once a document is index, all subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a document and then immediately reads it. The test works ok on a single node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here - https://gist.github.com/1221648. That class is all you need to run the test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel

Reply | Threaded
Open this post in threaded view
|

Re: Problem with realtime GET

Pavel Penchev
Superb!

Many thanks,
Pavel

On 18.09.2011 13:45, Shay Banon wrote:
Hey, thanks for the recreation. Managed to recreate it as well here, opened an issue and fixed it in 0.17 and master branch: https://github.com/elasticsearch/elasticsearch/issues/1344.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev <[hidden email]> wrote:
Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a project where I need to guarantee that once a document is index, all subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a document and then immediately reads it. The test works ok on a single node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here - https://gist.github.com/1221648. That class is all you need to run the test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel