Two indices on my document

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Two indices on my document

Praveen Kariyanahalli
Here is the problem I have.

I need to create an index on my document say by tenant name. And another one using say date. So if I need to lookup documents for a particular tenant .. I use just the first index. If I need to lookup documents for a particular tenant on a given date .. I use both. 

Can somebody point me to man pages as to how to achieve this ?  I am new to elastic search, if this has been answered already, please point me to that thread !

Thanks in Advance
pk
Reply | Threaded
Open this post in threaded view
|

Re: Two indices on my document

andym
Praveen,

For what you're asking sounds like you need just one ES index (unlike
traditional SQL you do not need explicitly specify multiple indexes
per column to speed up lookup, as the fields gets by indexed by
default in ES)

Given that you need one index, follow the instructions in "Getting
Started" section below
https://github.com/elasticsearch/elasticsearch

In the instructions substitute "user" for "tenant_name" and
"post_date" for "movein_date"
The "Getting Started" shows how to search per one field per query, if
you want to search per more then one field per query, you need to
combine your filters with AND or OR, instructions are here:
http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html

Having 2 different indices is certainly possibly, but it usually used
when different data is stored in the disctinct indices


On Jun 19, 1:28 pm, Praveen Kariyanahalli
<[hidden email]> wrote:

> Here is the problem I have.
>
> I need to create an index on my document say by tenant name. And another
> one using say date. So if I need to lookup documents for a particular
> tenant .. I use just the first index. If I need to lookup documents for a
> particular tenant on a given date .. I use both.
>
> Can somebody point me to man pages as to how to achieve this ?  I am new to
> elastic search, if this has been answered already, please point me to that
> thread !
>
> Thanks in Advance
> pk
Reply | Threaded
Open this post in threaded view
|

Re: Two indices on my document

Praveen Kariyanahalli
Hi Andy

Thanks for your quick response. I need date as an extra index because, if I need to delete an entire day's indexed data (say 6 month old data), it is just a single crontab entry doing an XDELETE on that index. In fact this is advised  in the elastic search tutorials, better way (more efficient way) than using TTL. That justifies having date as one index.

I need tenant name as another index for security purposes. I dont want to be querying for _all and then filtering on the tenant. 

Is there a place where ES multiple indexing is explained? 

Thanks in Advance
-pk 



On Tuesday, June 19, 2012 1:48:45 PM UTC-7, andym wrote:
Praveen,

For what you're asking sounds like you need just one ES index (unlike
traditional SQL you do not need explicitly specify multiple indexes
per column to speed up lookup, as the fields gets by indexed by
default in ES)

Given that you need one index, follow the instructions in "Getting
Started" section below
https://github.com/elasticsearch/elasticsearch

In the instructions substitute "user" for "tenant_name" and
"post_date" for "movein_date"
The "Getting Started" shows how to search per one field per query, if
you want to search per more then one field per query, you need to
combine your filters with AND or OR, instructions are here:
http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html

Having 2 different indices is certainly possibly, but it usually used
when different data is stored in the disctinct indices


On Jun 19, 1:28 pm, Praveen Kariyanahalli
<praveen.kariyanaha...@gmail.com> wrote:

> Here is the problem I have.
>
> I need to create an index on my document say by tenant name. And another
> one using say date. So if I need to lookup documents for a particular
> tenant .. I use just the first index. If I need to lookup documents for a
> particular tenant on a given date .. I use both.
>
> Can somebody point me to man pages as to how to achieve this ?  I am new to
> elastic search, if this has been answered already, please point me to that
> thread !
>
> Thanks in Advance
> pk
Reply | Threaded
Open this post in threaded view
|

Re: Two indices on my document

andym
Hi Praveen,

Using multiple indicies on the same server is no different then single
index, just prepand all your queries with index name (e.g. "index1/
search")

However if you want ES to do a cross-index SQL-like "join" for you on
tenant_id (assuming you want to have data structure like below), I am
not aware of such functionality -- you'd have to do "join" yourself.

index_master
{
    tenant_id
    tenant_name
}

index_month1
{
    tenant_id
    movein_date
    tenant_data
}

It's better is store your data in the form that will be to reflective
of your queries and avoid "join" (which might require data
duplication), which would mean adding  tenant_name into index_month1.
As to retrieving only the fields that are needed, you can specify the
fields that you want and not retrieve all the fields during your
query:

http://www.elasticsearch.org/guide/reference/api/search/fields.html

On Jun 19, 1:59 pm, Praveen Kariyanahalli
<[hidden email]> wrote:

> Hi Andy
>
> Thanks for your quick response. I need date as an extra index because, if I
> need to delete an entire day's indexed data (say 6 month old data), it is
> just a single crontab entry doing an XDELETE on that index. In fact this
> is advised  in the elastic search tutorials, better way (more efficient
> way) than using TTL. That justifies having date as one index.
>
> I need tenant name as another index for security purposes. I dont want to
> be querying for _all and then filtering on the tenant.
>
> Is there a place where ES multiple indexing is explained?
>
> Thanks in Advance
> -pk
>
>
>
>
>
>
>
> On Tuesday, June 19, 2012 1:48:45 PM UTC-7, andym wrote:
>
> > Praveen,
>
> > For what you're asking sounds like you need just one ES index (unlike
> > traditional SQL you do not need explicitly specify multiple indexes
> > per column to speed up lookup, as the fields gets by indexed by
> > default in ES)
>
> > Given that you need one index, follow the instructions in "Getting
> > Started" section below
> >https://github.com/elasticsearch/elasticsearch
>
> > In the instructions substitute "user" for "tenant_name" and
> > "post_date" for "movein_date"
> > The "Getting Started" shows how to search per one field per query, if
> > you want to search per more then one field per query, you need to
> > combine your filters with AND or OR, instructions are here:
> >http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html
>
> > Having 2 different indices is certainly possibly, but it usually used
> > when different data is stored in the disctinct indices
>
> > On Jun 19, 1:28 pm, Praveen Kariyanahalli
> > <[hidden email]> wrote:
> > > Here is the problem I have.
>
> > > I need to create an index on my document say by tenant name. And another
> > > one using say date. So if I need to lookup documents for a particular
> > > tenant .. I use just the first index. If I need to lookup documents for
> > a
> > > particular tenant on a given date .. I use both.
>
> > > Can somebody point me to man pages as to how to achieve this ?  I am new
> > to
> > > elastic search, if this has been answered already, please point me to
> > that
> > > thread !
>
> > > Thanks in Advance
> > > pk