Plugin for filtering results

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Plugin for filtering results

l3xan1
Hi,

I have to create my first ES Plugin. This is the purpose for the plugin:
1. Plugin receives search query and queries ES.
2. The results have, among other things, an ID. With this ID an Informix database has to be queried (to check if the user who initiated the search has sufficient rights to see the results - these rights can change often and depend on other stuff from the database, so it's not good to index those...) 
3. If one Item in the resultlist isn't allowed to be seen by the user it must be removed.
4. The results will be displayed.

As I said, I have no experience with the ES Plugin mechanism and this (http://jfarrell.github.io/) is the only tutorial I found, but it didn't help me very much (at least to understand the mechanism). 

Can anyone give me some good advice on this topic? Any Ideas how to realise it? Or does something like this already exist?

Thank You in advance 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

Alexander Reelsen-2
Hey,

if you ask more concrete questions, I'll be happy to help.

From your requirement point of view this sounds, as if you just need another terms filter. Maybe you do not need a plugin but rather a component which looks up these permissions up front and then makes the permissions part of the query? Might require a change in the component firing the query (but you could cache the permissions there as well I suppose). This also will lead to correct search and facet counts, which would be quite tedious if you would have to parse the result list everytime...


--Alex


On Thu, Jul 18, 2013 at 8:22 PM, l3xan1 <[hidden email]> wrote:
Hi,

I have to create my first ES Plugin. This is the purpose for the plugin:
1. Plugin receives search query and queries ES.
2. The results have, among other things, an ID. With this ID an Informix database has to be queried (to check if the user who initiated the search has sufficient rights to see the results - these rights can change often and depend on other stuff from the database, so it's not good to index those...) 
3. If one Item in the resultlist isn't allowed to be seen by the user it must be removed.
4. The results will be displayed.

As I said, I have no experience with the ES Plugin mechanism and this (http://jfarrell.github.io/) is the only tutorial I found, but it didn't help me very much (at least to understand the mechanism). 

Can anyone give me some good advice on this topic? Any Ideas how to realise it? Or does something like this already exist?

Thank You in advance 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

vineeth mohan
Like Alex said , there can be 2 approach
  1. Divide the access permission into 2 or N levels. Store the access permission level of each feed in the feed itself. When a query comes , map the username to the access permision and run a term query on it.
  2. Store all the user ID's which have access to the feed , in the feed itself. Again do a term query on this field.

Thanks

             Vineeth


On Fri, Jul 19, 2013 at 11:52 AM, Alexander Reelsen <[hidden email]> wrote:
Hey,

if you ask more concrete questions, I'll be happy to help.

From your requirement point of view this sounds, as if you just need another terms filter. Maybe you do not need a plugin but rather a component which looks up these permissions up front and then makes the permissions part of the query? Might require a change in the component firing the query (but you could cache the permissions there as well I suppose). This also will lead to correct search and facet counts, which would be quite tedious if you would have to parse the result list everytime...


--Alex


On Thu, Jul 18, 2013 at 8:22 PM, l3xan1 <[hidden email]> wrote:
Hi,

I have to create my first ES Plugin. This is the purpose for the plugin:
1. Plugin receives search query and queries ES.
2. The results have, among other things, an ID. With this ID an Informix database has to be queried (to check if the user who initiated the search has sufficient rights to see the results - these rights can change often and depend on other stuff from the database, so it's not good to index those...) 
3. If one Item in the resultlist isn't allowed to be seen by the user it must be removed.
4. The results will be displayed.

As I said, I have no experience with the ES Plugin mechanism and this (http://jfarrell.github.io/) is the only tutorial I found, but it didn't help me very much (at least to understand the mechanism). 

Can anyone give me some good advice on this topic? Any Ideas how to realise it? Or does something like this already exist?

Thank You in advance 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

l3xan1
In reply to this post by Alexander Reelsen-2
Hi,

thanks for your response.
Maybe I wasn't clear enough. It's not a simple UserID with permissions connected to it. It's more that every found document has a mark/ID and with this mark/ID (and the userid) the database has to be querried. So i'ts impossible to querry the database up front without results from the search (we're talking about a very(!) large database). The resultlist from ES is limited to a specific result size - so it's ok to go through (10 -20 Items) each time.


On Friday, 19 July 2013 08:22:31 UTC+2, Alexander Reelsen wrote:
Hey,

if you ask more concrete questions, I'll be happy to help.

From your requirement point of view this sounds, as if you just need another terms filter. Maybe you do not need a plugin but rather a component which looks up these permissions up front and then makes the permissions part of the query? Might require a change in the component firing the query (but you could cache the permissions there as well I suppose). This also will lead to correct search and facet counts, which would be quite tedious if you would have to parse the result list everytime...


--Alex


On Thu, Jul 18, 2013 at 8:22 PM, l3xan1 <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="jUOs2Gq5BnMJ">kno...@...> wrote:
Hi,

I have to create my first ES Plugin. This is the purpose for the plugin:
1. Plugin receives search query and queries ES.
2. The results have, among other things, an ID. With this ID an Informix database has to be queried (to check if the user who initiated the search has sufficient rights to see the results - these rights can change often and depend on other stuff from the database, so it's not good to index those...) 
3. If one Item in the resultlist isn't allowed to be seen by the user it must be removed.
4. The results will be displayed.

As I said, I have no experience with the ES Plugin mechanism and this (http://jfarrell.github.io/) is the only tutorial I found, but it didn't help me very much (at least to understand the mechanism). 

Can anyone give me some good advice on this topic? Any Ideas how to realise it? Or does something like this already exist?

Thank You in advance 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="jUOs2Gq5BnMJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

joergprante@gmail.com
In reply to this post by l3xan1
For this kind of use case, you don't need a plugin. Just fire up an ES
client, execute the query, iterate through the result hits and
optionally contact the database afterwards.

Jörg

Am 18.07.13 20:22, schrieb l3xan1:

> Hi,
>
> I have to create my first ES Plugin. This is the purpose for the plugin:
> 1. Plugin receives search query and queries ES.
> 2. The results have, among other things, an ID. With this ID an
> Informix database has to be queried (to check if the user who
> initiated the search has sufficient rights to see the results - these
> rights can change often and depend on other stuff from the database,
> so it's not good to index those...)
> 3. If one Item in the resultlist isn't allowed to be seen by the user
> it must be removed.
> 4. The results will be displayed.
>
> As I said, I have no experience with the ES Plugin mechanism and this
> (http://jfarrell.github.io/) is the only tutorial I found, but it
> didn't help me very much (at least to understand the mechanism).
>
> Can anyone give me some good advice on this topic? Any Ideas how to
> realise it? Or does something like this already exist?
>
> Thank You in advance
> --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [hidden email].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

l3xan1
Thats what I did up to now. Client received the results and sends the result to a server which does this filtering. The problem with this is that the client receives the result list.
I guess your approach is also that this filtering should be done on the client side. This isn't sufficient. The client could be able to see the non-filtered results. Thats why I thought about a plugin on the ES side, so that the result is the correct filtered version.

On Friday, 19 July 2013 09:20:54 UTC+2, Jörg Prante wrote:
For this kind of use case, you don't need a plugin. Just fire up an ES
client, execute the query, iterate through the result hits and
optionally contact the database afterwards.

Jörg

Am 18.07.13 20:22, schrieb l3xan1:

> Hi,
>
> I have to create my first ES Plugin. This is the purpose for the plugin:
> 1. Plugin receives search query and queries ES.
> 2. The results have, among other things, an ID. With this ID an
> Informix database has to be queried (to check if the user who
> initiated the search has sufficient rights to see the results - these
> rights can change often and depend on other stuff from the database,
> so it's not good to index those...)
> 3. If one Item in the resultlist isn't allowed to be seen by the user
> it must be removed.
> 4. The results will be displayed.
>
> As I said, I have no experience with the ES Plugin mechanism and this
> (http://jfarrell.github.io/) is the only tutorial I found, but it
> didn't help me very much (at least to understand the mechanism).
>
> Can anyone give me some good advice on this topic? Any Ideas how to
> realise it? Or does something like this already exist?
>
> Thank You in advance
> --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="uxlomcxS0YMJ">elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Plugin for filtering results

joergprante@gmail.com
No, I mean that is a task for implementing an ES client module, at a single, secure place in a proxy, between front end and back end. This is much easier than a ES plugin. Imagine multi node ES cluster. Why should you have to install the plugin at each ES node, and why should each ES node connect to the database? You would have to protect each node which is a quite a challenge for administration.


On Fri, Jul 19, 2013 at 9:38 AM, l3xan1 <[hidden email]> wrote:
Thats what I did up to now. Client received the results and sends the result to a server which does this filtering. The problem with this is that the client receives the result list.
I guess your approach is also that this filtering should be done on the client side. This isn't sufficient. The client could be able to see the non-filtered results. Thats why I thought about a plugin on the ES side, so that the result is the correct filtered version.


On Friday, 19 July 2013 09:20:54 UTC+2, Jörg Prante wrote:
For this kind of use case, you don't need a plugin. Just fire up an ES
client, execute the query, iterate through the result hits and
optionally contact the database afterwards.

Jörg

Am 18.07.13 20:22, schrieb l3xan1:

> Hi,
>
> I have to create my first ES Plugin. This is the purpose for the plugin:
> 1. Plugin receives search query and queries ES.
> 2. The results have, among other things, an ID. With this ID an
> Informix database has to be queried (to check if the user who
> initiated the search has sufficient rights to see the results - these
> rights can change often and depend on other stuff from the database,
> so it's not good to index those...)
> 3. If one Item in the resultlist isn't allowed to be seen by the user
> it must be removed.
> 4. The results will be displayed.
>
> As I said, I have no experience with the ES Plugin mechanism and this
> (http://jfarrell.github.io/) is the only tutorial I found, but it
> didn't help me very much (at least to understand the mechanism).
>
> Can anyone give me some good advice on this topic? Any Ideas how to
> realise it? Or does something like this already exist?
>
> Thank You in advance
> --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elasticsearc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.