Document Level Permissions Filtering

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Document Level Permissions Filtering

richardwhatever
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

vineeth mohan
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth

On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.

Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Michael Sick
Shay,

Building on this question, are you considering adding an internal security model to ES? 

If so, have you looked at what was done in the Apache Accumulo project? Short story, this was a Big Table implementation built out by the US Intelligence Community and built out a finer grained security model that still scaled. If you are considering adding ES security, I'm curious if you think this level of granularity is desirable/feasible.
 
http://incubator.apache.org/accumulo/user_manual_1.4-incubating/Security.html 

--Mike

On Fri, Mar 23, 2012 at 2:50 PM, Vineeth Mohan <[hidden email]> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth

On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.


Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

joergprante@gmail.com
Security in search engines is a wide subject. 

The "small solution" is just to hand over authorization tokens, index them, and later add filters to all queries automatically. This could be done by a web front-end proxy placed before the Elasticsearch index/search REST API.

I think the most preferred way to have document-level security would be a "big solution", i.e. writing an Elasticsearch security plugin with the following features included

- transport layer security between client and server (TLS)
- server awareness of client identity (establish ACL/role based security)
- secure indexing: only allow authorized clients to write documents (perform ACL/role-based checks)
- secure query: index visibility, document visibility, filter query results due to client authorization
- enforce document / index data integrity (checksums, end-to-end security)
- constraint violation alerting
- and, really advanced stuff, adjust highlighting to filter out sensitive (restricted) fields (privacy)
- even more "paranoid" stuff

Most of the work could be taken over by front-end web processes, and the Elasticsearch security plugin could offer some conventions for a security model, i.e. rules for where index/document-level TLS/ACL/Role stuff is stored and how it is used by clients via a REST API.

Best regards,

Jörg

Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

kimchy
Administrator
Agreed, security / ACL is a big topic. For document level one, the best way is to associate the relevant data with each document, and filter by it.

On Sun, Mar 25, 2012 at 1:46 PM, Jörg Prante <[hidden email]> wrote:
Security in search engines is a wide subject. 

The "small solution" is just to hand over authorization tokens, index them, and later add filters to all queries automatically. This could be done by a web front-end proxy placed before the Elasticsearch index/search REST API.

I think the most preferred way to have document-level security would be a "big solution", i.e. writing an Elasticsearch security plugin with the following features included

- transport layer security between client and server (TLS)
- server awareness of client identity (establish ACL/role based security)
- secure indexing: only allow authorized clients to write documents (perform ACL/role-based checks)
- secure query: index visibility, document visibility, filter query results due to client authorization
- enforce document / index data integrity (checksums, end-to-end security)
- constraint violation alerting
- and, really advanced stuff, adjust highlighting to filter out sensitive (restricted) fields (privacy)
- even more "paranoid" stuff

Most of the work could be taken over by front-end web processes, and the Elasticsearch security plugin could offer some conventions for a security model, i.e. rules for where index/document-level TLS/ACL/Role stuff is stored and how it is used by clients via a REST API.

Best regards,

Jörg


Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

richardwhatever
In reply to this post by vineeth mohan
Hi Vineeth,

Sorry for delay in response, i was out of the country.

My requirements are not about filtering the incoming search queries, but rather about restricting the indexed documents to ones a user has permission to search... i think it has to be done by the search engine or paging etc would not work properly.

Regards,

Richard

On 23 March 2012 18:50, Vineeth Mohan <[hidden email]> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth


On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.




--
Richard Brown
@richardwhatever

Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

kimchy
Administrator
If you add relevant security aspects to the documents indexed, you can filter by them.

On Thu, Apr 5, 2012 at 9:17 PM, Richard Brown <[hidden email]> wrote:
Hi Vineeth,

Sorry for delay in response, i was out of the country.

My requirements are not about filtering the incoming search queries, but rather about restricting the indexed documents to ones a user has permission to search... i think it has to be done by the search engine or paging etc would not work properly.

Regards,

Richard


On 23 March 2012 18:50, Vineeth Mohan <[hidden email]> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth


On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.




--
Richard Brown
@richardwhatever


Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Yann Barraud
In reply to this post by richardwhatever
Hi Richard & all,
 
Did you find a solution for this problem ? I have quite the same requirments/ questions about differents sources (out of ManifoldCF that seems to manage the whole bunch)....
 
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,

Sorry for delay in response, i was out of the country.

My requirements are not about filtering the incoming search queries, but rather about restricting the indexed documents to ones a user has permission to search... i think it has to be done by the search engine or paging etc would not work properly.

Regards,

Richard

On 23 March 2012 18:50, Vineeth Mohan <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="FQLfZ2TzgMQJ">vineet...@...> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth


On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="FQLfZ2TzgMQJ">richard...@...> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.




--
Richard Brown
@richardwhatever

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

richardwhatever
In reply to this post by richardwhatever
Hi Yann,

No, I didn't implement a Solr Solution as yet. Currently looking into using RavenDB instead as its based on lucene.net, but with security built in.

Richard


On 21 November 2012 10:57, Yann Barraud <[hidden email]> wrote:
Hi Richard & all,
 
Did you find a solution for this problem ? I have quite the same requirments/ questions about differents sources (out of ManifoldCF that seems to manage the whole bunch)....
 
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,

Sorry for delay in response, i was out of the country.

My requirements are not about filtering the incoming search queries, but rather about restricting the indexed documents to ones a user has permission to search... i think it has to be done by the search engine or paging etc would not work properly.

Regards,

Richard

On 23 March 2012 18:50, Vineeth Mohan <[hidden email]> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth


On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.




--
Richard Brown
@richardwhatever

--
 
 



--
Richard Brown
@richardwhatever

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Lukáš Vlček

Hi,

May be not useful idea in your particular use case but in many organizations document-user visibility is driven by organization structure/groups, i.e. not really at the level of individual documents. If that would fit to your situation then you can consider indexing documents into indices by "organization departments" and then allow users to search across relevant indices only. Also index aliases could be useful in this scenario.

Just an idea.

Regards,
Lukáš

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Hendrik
In reply to this post by joergprante@gmail.com
Started this to fulfill our company needs:
https://github.com/salyh/elasticsearch-security-plugin

Am Sonntag, 25. März 2012 13:46:53 UTC+2 schrieb Jörg Prante:
Security in search engines is a wide subject. 

The "small solution" is just to hand over authorization tokens, index them, and later add filters to all queries automatically. This could be done by a web front-end proxy placed before the Elasticsearch index/search REST API.

I think the most preferred way to have document-level security would be a "big solution", i.e. writing an Elasticsearch security plugin with the following features included

- transport layer security between client and server (TLS)
- server awareness of client identity (establish ACL/role based security)
- secure indexing: only allow authorized clients to write documents (perform ACL/role-based checks)
- secure query: index visibility, document visibility, filter query results due to client authorization
- enforce document / index data integrity (checksums, end-to-end security)
- constraint violation alerting
- and, really advanced stuff, adjust highlighting to filter out sensitive (restricted) fields (privacy)
- even more "paranoid" stuff

Most of the work could be taken over by front-end web processes, and the Elasticsearch security plugin could offer some conventions for a security model, i.e. rules for where index/document-level TLS/ACL/Role stuff is stored and how it is used by clients via a REST API.

Best regards,

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

joergprante@gmail.com
Great effort. Do you plan to add JAAS?

Cheers,

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Hendrik
yes (via tomcat), but i guess the focus will be on kerberos.
My long term plan is to add also things like:

- encryption for fs index store
- stuff like https://github.com/texeltek/elasticsearch-accumulo-security

Am Donnerstag, 31. Oktober 2013 14:59:25 UTC+1 schrieb Jörg Prante:
Great effort. Do you plan to add JAAS?

Cheers,

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Hendrik
there is some progress here: https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/tavroa3Nw5g

Am Mittwoch, 6. November 2013 12:34:07 UTC+1 schrieb Hendrik:
yes (via tomcat), but i guess the focus will be on kerberos.
My long term plan is to add also things like:

- encryption for fs index store
- stuff like https://github.com/texeltek/elasticsearch-accumulo-security

Am Donnerstag, 31. Oktober 2013 14:59:25 UTC+1 schrieb Jörg Prante:
Great effort. Do you plan to add JAAS?

Cheers,

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Document Level Permissions Filtering

Hendrik
In reply to this post by richardwhatever
Maybe this is interesting for you https://github.com/salyh/elasticsearch-security-plugin

Am Donnerstag, 22. November 2012 18:37:05 UTC+1 schrieb Richard Brown:
Hi Yann,

No, I didn't implement a Solr Solution as yet. Currently looking into using RavenDB instead as its based on <a href="http://lucene.net" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Flucene.net\46sa\75D\46sntz\0751\46usg\75AFQjCNFXiBf87dKahsu58ukciI0bVBSwJw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Flucene.net\46sa\75D\46sntz\0751\46usg\75AFQjCNFXiBf87dKahsu58ukciI0bVBSwJw';return true;">lucene.net, but with security built in.

Richard


On 21 November 2012 10:57, Yann Barraud <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="ZWDQjOSBBuMJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">yann.b...@...> wrote:
Hi Richard & all,
 
Did you find a solution for this problem ? I have quite the same requirments/ questions about differents sources (out of ManifoldCF that seems to manage the whole bunch)....
 
Thanks.
Yann
Le jeudi 5 avril 2012 20:17:09 UTC+2, Richard Brown a écrit :
Hi Vineeth,

Sorry for delay in response, i was out of the country.

My requirements are not about filtering the incoming search queries, but rather about restricting the indexed documents to ones a user has permission to search... i think it has to be done by the search engine or paging etc would not work properly.

Regards,

Richard

On 23 March 2012 18:50, Vineeth Mohan <[hidden email]> wrote:
you will have to implement that on proxy level or something.
Like you need to set a permission level on each document or type or even index.
And when a user query comes , incercept the query in between using a proxy or something , see if the user have permission and then only let it through.

Thanks
           Vineeth


On Wed, Mar 21, 2012 at 5:53 PM, Richard Brown <[hidden email]> wrote:
Hi,

I'm looking for a search engine that provides document level access / filtering. I had looked at ManifoldCF with Solr as one option.  Does elasticsearch offer this kind of functionality?  I need to be able to restrict search results in a granual way, whereby each document can have permissions set for each potential user of the system.

Thanks,

Richard.




--
Richard Brown
@richardwhatever
<a href="http://uk.linkedin.com/in/richardbrownni" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fuk.linkedin.com%2Fin%2Frichardbrownni\46sa\75D\46sntz\0751\46usg\75AFQjCNEREN2__mpzldON2zwu0O0HZrO1Rw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fuk.linkedin.com%2Fin%2Frichardbrownni\46sa\75D\46sntz\0751\46usg\75AFQjCNEREN2__mpzldON2zwu0O0HZrO1Rw';return true;">uk.linkedin.com/in/richardbrownni

--
 
 



--
Richard Brown
@richardwhatever
<a href="http://uk.linkedin.com/in/richardbrownni" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fuk.linkedin.com%2Fin%2Frichardbrownni\46sa\75D\46sntz\0751\46usg\75AFQjCNEREN2__mpzldON2zwu0O0HZrO1Rw';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fuk.linkedin.com%2Fin%2Frichardbrownni\46sa\75D\46sntz\0751\46usg\75AFQjCNEREN2__mpzldON2zwu0O0HZrO1Rw';return true;">uk.linkedin.com/in/richardbrownni

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b2747756-f386-4118-a8e3-6489be218a36%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.