what would be the effect of using an arbitrary large count

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

what would be the effect of using an arbitrary large count

Ron Sher
Hi
We have a multi tenant SaaS solution and we expose our use of elasticsearch through an API. We noticed that clients who use the API directly, and not through our front-end, don't bother with paging and just use a count of 500k or 1m even if the actual count is much lower. I was wondering about the effect of not limiting the count since we're thinking of making a breaking change and limit this to 100.

Thanks for your help
Ron

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9507d1c8-5f32-41bf-a537-3cd4e0c2ba9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: what would be the effect of using an arbitrary large count

kazoompa-3
I am still not expert an in ES but surely when not paging the process time will be higher because more documents have to be brought back in the response. However, depending on what kind of queries you perform, the subsequent queries will be faster. I am thinking of filter bool queries with caching and bitset query representation. I doubt that a size of 1m will have any effect if there are only 100 documents indexed.

I wouldn't advise removing the paging for the sake of bandwidth and network traffic especially if you have many clients using the same server, may be a very optimized server setup can help.

We do a full query only when doing statistical calculation requiring all documents.





On Wednesday, December 3, 2014 11:40:05 PM UTC-5, Ron Sher wrote:
Hi
We have a multi tenant SaaS solution and we expose our use of elasticsearch through an API. We noticed that clients who use the API directly, and not through our front-end, don't bother with paging and just use a count of 500k or 1m even if the actual count is much lower. I was wondering about the effect of not limiting the count since we're thinking of making a breaking change and limit this to 100.

Thanks for your help
Ron

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51e07a71-3649-4e99-9217-21b0f32b20b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.