Finding out if an index exists

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Finding out if an index exists

zohar
Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

James Cook
We do this:

var establishedIndicies = [];

function initIndex(index) { if (establishedIndicies.contains(index)) { return; } try { getClient().admin().indices().status(indicesStatusRequest(index)).actionGet(); } catch(e) { // Index does not exist yet. getClient().admin().indices().create(createIndexRequest(index)).actionGet(); establishedIndicies.push(index); } }

-- jim

On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:

Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-index-exists-tp1820921p1820921.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

James Cook
In reply to this post by zohar
The mail server must be slow. I just got your message now.

-- jim

On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:

Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-index-exists-tp1820921p1820921.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

kimchy
Administrator
In reply to this post by James Cook
The cluster state is the way to do it (preferable over the index status API). Use the builder API which is nicer (prepareState()).

On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]> wrote:
We do this:

var establishedIndicies = [];

function initIndex(index) { if (establishedIndicies.contains(index)) { return; } try { getClient().admin().indices().status(indicesStatusRequest(index)).actionGet(); } catch(e) { // Index does not exist yet. getClient().admin().indices().create(createIndexRequest(index)).actionGet(); establishedIndicies.push(index); } }

-- jim

On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:

Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-index-exists-tp1820921p1820921.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

Paul Loy
I find that if I call this too soon after starting ES my index does not show up. Is there any way to determine when the cluster is in a state where it knows about the indices, even if it has not yet fully loaded them from a gateway?

On Tue, Nov 2, 2010 at 2:20 PM, Shay Banon <[hidden email]> wrote:
The cluster state is the way to do it (preferable over the index status API). Use the builder API which is nicer (prepareState()).

On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]> wrote:
We do this:

var establishedIndicies = [];

function initIndex(index) { if (establishedIndicies.contains(index)) { return; } try { getClient().admin().indices().status(indicesStatusRequest(index)).actionGet(); } catch(e) { // Index does not exist yet. getClient().admin().indices().create(createIndexRequest(index)).actionGet(); establishedIndicies.push(index); } }

-- jim

On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:

Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-index-exists-tp1820921p1820921.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.





--
---------------------------------------------
Paul Loy
[hidden email]
http://uk.linkedin.com/in/paulloy
Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

kimchy
Administrator
Use the cluster state API, its the first thing that gets recovered from the gateway. While it has not recovered yet, its is in a blocked state, and you can check that by using ClusterState#getBlocks() and verifying that there are no global blocks.

Another option to wait for the cluster to recover is the use the cluster health API, with all the different waitFor options. Waiting for a yellow status on a specific index or across all indices.

-shay.banon

On Wed, Nov 3, 2010 at 1:23 AM, Paul Loy <[hidden email]> wrote:
I find that if I call this too soon after starting ES my index does not show up. Is there any way to determine when the cluster is in a state where it knows about the indices, even if it has not yet fully loaded them from a gateway?


On Tue, Nov 2, 2010 at 2:20 PM, Shay Banon <[hidden email]> wrote:
The cluster state is the way to do it (preferable over the index status API). Use the builder API which is nicer (prepareState()).

On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]> wrote:
We do this:

var establishedIndicies = [];

function initIndex(index) { if (establishedIndicies.contains(index)) { return; } try { getClient().admin().indices().status(indicesStatusRequest(index)).actionGet(); } catch(e) { // Index does not exist yet. getClient().admin().indices().create(createIndexRequest(index)).actionGet(); establishedIndicies.push(index); } }

-- jim

On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:

Hi
We have some code to detect if an index exists :

client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(index.name())

is there a slightly more compact/simpler way to do this ?

Cheers
Z
--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-index-exists-tp1820921p1820921.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.





--
---------------------------------------------
Paul Loy
[hidden email]
http://uk.linkedin.com/in/paulloy

Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

Christian Pesch
In reply to this post by kimchy
Hi Shay,

I've stumbled across a problem with 0.12.1 and node.local=true when
checking for the existance of an index.
I'm using the sequence

 
client.admin().cluster().prepareHealth().setWaitForYellowStatus().execute().actionGet();
        ClusterStateResponse response =
client.admin().cluster().prepareState().execute().actionGet();
        boolean hasIndex =
response.getState().metaData().hasIndex(indexName);
        if(!hasIndex)
 
client.admin().indices().create(createIndexRequest(indexName)).actionGet();

This works fine when I set discovery.zen.ping.unicast.hosts but fails
1 or 2 times out of 10 when setting
node.local=true. Then setWaitForYellowStatus() does not guarantee that
all indices are known and
the hasIndex() returns false although the index exists (and is visible
after a few more seconds).
Is this a known problem?

Kind regards
Christian

On Nov 2, 10:20 pm, Shay Banon <[hidden email]> wrote:

> The cluster state is the way to do it (preferable over the index status
> API). Use the builder API which is nicer (prepareState()).
>
> On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]> wrote:
> > We do this:
>
> > var establishedIndicies = [];
>
> > function initIndex(index) { if (establishedIndicies.contains(index)) {
> > return; } try {
> > getClient().admin().indices().status(indicesStatusRequest(index)).actionGet();
> > } catch(e) { // Index does not exist yet.
> > getClient().admin().indices().create(createIndexRequest(index)).actionGet();
> > establishedIndicies.push(index); } }
>
> > -- jim
>
> > On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:
>
> >> Hi
> >> We have some code to detect if an index exists :
>
> >> client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(
> >> index.name())
>
> >> is there a slightly more compact/simpler way to do this ?
>
> >> Cheers
> >> Z
> >> --
> >> View this message in context:
> >>http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-ind...
> >> Sent from the ElasticSearch Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

zohar
We  have the very same problem
Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

kimchy
Administrator
In reply to this post by Christian Pesch
This should be fixed in master (0.13), can you give it a go?

On Mon, Nov 15, 2010 at 6:47 PM, Christian Pesch <[hidden email]> wrote:
Hi Shay,

I've stumbled across a problem with 0.12.1 and node.local=true when
checking for the existance of an index.
I'm using the sequence


client.admin().cluster().prepareHealth().setWaitForYellowStatus().execute().actionGet();
       ClusterStateResponse response =
client.admin().cluster().prepareState().execute().actionGet();
       boolean hasIndex =
response.getState().metaData().hasIndex(indexName);
       if(!hasIndex)

client.admin().indices().create(createIndexRequest(indexName)).actionGet();

This works fine when I set discovery.zen.ping.unicast.hosts but fails
1 or 2 times out of 10 when setting
node.local=true. Then setWaitForYellowStatus() does not guarantee that
all indices are known and
the hasIndex() returns false although the index exists (and is visible
after a few more seconds).
Is this a known problem?

Kind regards
Christian

On Nov 2, 10:20 pm, Shay Banon <[hidden email]> wrote:
> The cluster state is the way to do it (preferable over the index status
> API). Use the builder API which is nicer (prepareState()).
>
> On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]> wrote:
> > We do this:
>
> > var establishedIndicies = [];
>
> > function initIndex(index) { if (establishedIndicies.contains(index)) {
> > return; } try {
> > getClient().admin().indices().status(indicesStatusRequest(index)).actionGet();
> > } catch(e) { // Index does not exist yet.
> > getClient().admin().indices().create(createIndexRequest(index)).actionGet();
> > establishedIndicies.push(index); } }
>
> > -- jim
>
> > On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:
>
> >> Hi
> >> We have some code to detect if an index exists :
>
> >> client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(
> >> index.name())
>
> >> is there a slightly more compact/simpler way to do this ?
>
> >> Cheers
> >> Z
> >> --
> >> View this message in context:
> >>http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-ind...
> >> Sent from the ElasticSearch Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

Christian Pesch
Great! The error didn't occur on 15 startups, it seems to be gone.
Are you going to release 0.13.0 at the end of November?

On Nov 15, 5:57 pm, Shay Banon <[hidden email]> wrote:

> This should be fixed in master (0.13), can you give it a go?
>
> On Mon, Nov 15, 2010 at 6:47 PM, Christian Pesch <[hidden email]> wrote:
> > Hi Shay,
>
> > I've stumbled across a problem with 0.12.1 and node.local=true when
> > checking for the existance of an index.
> > I'm using the sequence
>
> > client.admin().cluster().prepareHealth().setWaitForYellowStatus().execute().actionGet();
> >        ClusterStateResponse response =
> > client.admin().cluster().prepareState().execute().actionGet();
> >        boolean hasIndex =
> > response.getState().metaData().hasIndex(indexName);
> >        if(!hasIndex)
>
> > client.admin().indices().create(createIndexRequest(indexName)).actionGet();
>
> > This works fine when I set discovery.zen.ping.unicast.hosts but fails
> > 1 or 2 times out of 10 when setting
> > node.local=true. Then setWaitForYellowStatus() does not guarantee that
> > all indices are known and
> > the hasIndex() returns false although the index exists (and is visible
> > after a few more seconds).
> > Is this a known problem?
>
> > Kind regards
> > Christian
>
> > On Nov 2, 10:20 pm, Shay Banon <[hidden email]> wrote:
> > > The cluster state is the way to do it (preferable over the index status
> > > API). Use the builder API which is nicer (prepareState()).
>
> > > On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]>
> > wrote:
> > > > We do this:
>
> > > > var establishedIndicies = [];
>
> > > > function initIndex(index) { if (establishedIndicies.contains(index)) {
> > > > return; } try {
>
> > getClient().admin().indices().status(indicesStatusRequest(index)).actionGet();
> > > > } catch(e) { // Index does not exist yet.
>
> > getClient().admin().indices().create(createIndexRequest(index)).actionGet();
> > > > establishedIndicies.push(index); } }
>
> > > > -- jim
>
> > > > On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:
>
> > > >> Hi
> > > >> We have some code to detect if an index exists :
>
> > client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(
> > > >> index.name())
>
> > > >> is there a slightly more compact/simpler way to do this ?
>
> > > >> Cheers
> > > >> Z
> > > >> --
> > > >> View this message in context:
> > > >>http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-ind.
> > ..
> > > >> Sent from the ElasticSearch Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Finding out if an index exists

kimchy
Administrator
0.13 will be released this week :), finalizing things now and running the "long running" regression tests...

On Mon, Nov 15, 2010 at 7:27 PM, Christian Pesch <[hidden email]> wrote:
Great! The error didn't occur on 15 startups, it seems to be gone.
Are you going to release 0.13.0 at the end of November?

On Nov 15, 5:57 pm, Shay Banon <[hidden email]> wrote:
> This should be fixed in master (0.13), can you give it a go?
>
> On Mon, Nov 15, 2010 at 6:47 PM, Christian Pesch <[hidden email]> wrote:
> > Hi Shay,
>
> > I've stumbled across a problem with 0.12.1 and node.local=true when
> > checking for the existance of an index.
> > I'm using the sequence
>
> > client.admin().cluster().prepareHealth().setWaitForYellowStatus().execute().actionGet();
> >        ClusterStateResponse response =
> > client.admin().cluster().prepareState().execute().actionGet();
> >        boolean hasIndex =
> > response.getState().metaData().hasIndex(indexName);
> >        if(!hasIndex)
>
> > client.admin().indices().create(createIndexRequest(indexName)).actionGet();
>
> > This works fine when I set discovery.zen.ping.unicast.hosts but fails
> > 1 or 2 times out of 10 when setting
> > node.local=true. Then setWaitForYellowStatus() does not guarantee that
> > all indices are known and
> > the hasIndex() returns false although the index exists (and is visible
> > after a few more seconds).
> > Is this a known problem?
>
> > Kind regards
> > Christian
>
> > On Nov 2, 10:20 pm, Shay Banon <[hidden email]> wrote:
> > > The cluster state is the way to do it (preferable over the index status
> > > API). Use the builder API which is nicer (prepareState()).
>
> > > On Tue, Nov 2, 2010 at 10:34 PM, James Cook <[hidden email]>
> > wrote:
> > > > We do this:
>
> > > > var establishedIndicies = [];
>
> > > > function initIndex(index) { if (establishedIndicies.contains(index)) {
> > > > return; } try {
>
> > getClient().admin().indices().status(indicesStatusRequest(index)).actionGet();
> > > > } catch(e) { // Index does not exist yet.
>
> > getClient().admin().indices().create(createIndexRequest(index)).actionGet();
> > > > establishedIndicies.push(index); } }
>
> > > > -- jim
>
> > > > On Mon, Nov 1, 2010 at 6:12 AM, zohar <[hidden email]> wrote:
>
> > > >> Hi
> > > >> We have some code to detect if an index exists :
>
> > client.admin().cluster().state(clusterStateRequest()).actionGet().getState().metaData().getIndices().containsKey(
> > > >> index.name())
>
> > > >> is there a slightly more compact/simpler way to do this ?
>
> > > >> Cheers
> > > >> Z
> > > >> --
> > > >> View this message in context:
> > > >>http://elasticsearch-users.115913.n3.nabble.com/Finding-out-if-an-ind.
> > ..
> > > >> Sent from the ElasticSearch Users mailing list archive at Nabble.com.