Multi-tenacy

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Multi-tenacy

Michael-7
Hi,

I am considering using elastic search in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike
Reply | Threaded
Open this post in threaded view
|

Re: Multi-tenacy

Berkay Mollamustafaoglu-2

Hi Mike, 

Multi tenancy is mentioned here in the doc in you haven't seen it already

API makes it easy to work with that type of configuration. I can't comment on the impact of having thousands of indices, but if the twitter example is any indication, this should not be a problem.

Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype


On Wed, Mar 24, 2010 at 12:56 PM, Michael <[hidden email]> wrote:
Hi,

I am considering using elastic search in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike

Reply | Threaded
Open this post in threaded view
|

Re: Multi-tenacy

kimchy
Administrator
Hi,

  Multiple indices support is basically bounded by memory on each node/server. Each node in the cluster holds the cluster state (and the derived state from it, for example, a parsed version of the mapping definition). The cluster state includes all the indices, their mappings and so on. To be honest, I have not tried to profile and see what the impact of having thousands of indices, but the system is certainly built to handle it if memory permits. Moreover, its going to be different from one app to the other, since, for example, the mappings used (or automatically created) affect the final overhead of each index within the cluster state.

   I do know of places that can be optimized to retain less memory, but thats the perfect case of premature optimization. What I would suggest is, if you can, run a simple test, with the amount of servers, and see how it holds. I am certainly here to help and optimize if needed. I do plan to run a similar test as soon as I can.

  In terms of scaling, assuming each node has enough memory to store the cluster state, then yes, the more nodes you will add to the system, the better.

  Last, as for security, there isn't one built into elasticsearch, so you will have to build it on the client side. I definitely plan to add security features to elasticsearch, as multi tenancy demands it, but its not going to be soon (i.e. in the next couple of months). As a side node, see Hadoop for example, they just started to work on a security model :), and they are the perfect example for multi tenancy.

-shay.banon

On Wed, Mar 24, 2010 at 7:47 PM, Berkay Mollamustafaoglu <[hidden email]> wrote:

Hi Mike, 

Multi tenancy is mentioned here in the doc in you haven't seen it already

API makes it easy to work with that type of configuration. I can't comment on the impact of having thousands of indices, but if the twitter example is any indication, this should not be a problem.

Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype



On Wed, Mar 24, 2010 at 12:56 PM, Michael <[hidden email]> wrote:
Hi,

I am considering using elastic search in a multi-tenant application,
should I have any concerns using an index per customer? Will thousands
of seperate indexes cause any problems? Other than those scaling can
fix?

Also can I secure each index separately?

Any documentation/samples for multi-tenacy.

Best.

Mike