Test data for elasticsearch-head

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Test data for elasticsearch-head

mobz-2
Hi All,
  I'm the main developer behind elasticseach-head, and I have a plee.

   I don't have a lot of time to work on this project, but I _do_ want to keep moving it forward - However, I'm not a java guy, and I'm not a search guy and I don't (anymore) work on elasticsearch day-to-day, but I am a UI guy. What I need to enable me to add features, fix bugs and feel confident about merging pull requests, is sample data. The ability to quickly spin up a cluster that has percolation, or makes extensive use of the _parent feature or has 100 indices. 

So my question: does anyone have or would be willing to contribute shell scripts (or something else) that create indices and clusters with various elasticsearch features, field types, mappings, percolation, rivers.

For example, I would love to add support for rivers to es-head, but I don't want to have to _learn_ about the intricacies of each river, it will take too much time. If there are some scripts out there already to start up a mongodb db, and start a river, then I could probably add an interface pretty quickly.

My existing sample directory https://github.com/mobz/elasticsearch-head/tree/master/lib/es/test/index_generators is, however woefully small . I would accept pull requests against this, or perhaps there is an existing resource out there. Maybe this would be a great resource for more people that could be a standalone project? Maybe kimchy has a giant stash?!

-B

Reply | Threaded
Open this post in threaded view
|

Re: Test data for elasticsearch-head

jagdeep
Thanks for the help and support. I will share the scripts with you

Regards
Jagdeep

On May 4, 6:27 am, mobz <[hidden email]> wrote:

> Hi All,
>   I'm the main developer behind elasticseach-head, and I have a plee.
>
>    I don't have a lot of time to work on this project, but I _do_ want to
> keep moving it forward - However, I'm not a java guy, and I'm not a search
> guy and I don't (anymore) work on elasticsearch day-to-day, but I am a UI
> guy. What I need to enable me to add features, fix bugs and feel confident
> about merging pull requests, is sample data. The ability to quickly spin up
> a cluster that has percolation, or makes extensive use of the _parent
> feature or has 100 indices.
>
> So my question: does anyone have or would be willing to contribute shell
> scripts (or something else) that create indices and clusters with various
> elasticsearch features, field types, mappings, percolation, rivers.
>
> For example, I would love to add support for rivers to es-head, but I don't
> want to have to _learn_ about the intricacies of each river, it will take
> too much time. If there are some scripts out there already to start up a
> mongodb db, and start a river, then I could probably add an interface
> pretty quickly.
>
> My existing sample directoryhttps://github.com/mobz/elasticsearch-head/tree/master/lib/es/test/in...
> is, however woefully small . I would accept pull requests against this, or
> perhaps there is an existing resource out there. Maybe this would be a
> great resource for more people that could be a standalone project? Maybe
> kimchy has a giant stash?!
>
> -B
Reply | Threaded
Open this post in threaded view
|

Re: Test data for elasticsearch-head

Otis Gospodnetic
In reply to this post by mobz-2
This may be of some help:

https://github.com/sematext/ActionGenerator 

It's not really documented and needs to be extended, but it does work.  Basically, while developing SPM for Solr/ElasticSearch/HBase/Sensei we needed something like you - something that could populate indices or DB and that could query it.  So that is what ActionGenerator does.  And the idea is that its core is generic and extensible, so that people can plug in action generators for their specific systems.

You or others here may want to fork, add ElasticSearch action generator, and we'll be happy to merge all valuable contributions.  We may actually have an ES-specific action generator already, just not committed there yet.  I can check if this is something you want to use?

Anyhow, maybe this can be of help.

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Scalable Performance Monitoring - http://sematext.com/spm/index.html


On Thursday, May 3, 2012 9:27:39 PM UTC-4, mobz wrote:
Hi All,
  I'm the main developer behind elasticseach-head, and I have a plee.

   I don't have a lot of time to work on this project, but I _do_ want to keep moving it forward - However, I'm not a java guy, and I'm not a search guy and I don't (anymore) work on elasticsearch day-to-day, but I am a UI guy. What I need to enable me to add features, fix bugs and feel confident about merging pull requests, is sample data. The ability to quickly spin up a cluster that has percolation, or makes extensive use of the _parent feature or has 100 indices. 

So my question: does anyone have or would be willing to contribute shell scripts (or something else) that create indices and clusters with various elasticsearch features, field types, mappings, percolation, rivers.

For example, I would love to add support for rivers to es-head, but I don't want to have to _learn_ about the intricacies of each river, it will take too much time. If there are some scripts out there already to start up a mongodb db, and start a river, then I could probably add an interface pretty quickly.

My existing sample directory https://github.com/mobz/elasticsearch-head/tree/master/lib/es/test/index_generators is, however woefully small . I would accept pull requests against this, or perhaps there is an existing resource out there. Maybe this would be a great resource for more people that could be a standalone project? Maybe kimchy has a giant stash?!

-B

Reply | Threaded
Open this post in threaded view
|

Re: Test data for elasticsearch-head

Rafał Kuć-3
In reply to this post by mobz-2
Hi,

 Keep an eye on Sematext blog, we will be blogging about those ActionGenerator Otis mentioned soon :)

Regards,
Rafał

W dniu piątek, 4 maja 2012 03:27:39 UTC+2 użytkownik mobz napisał:
Hi All,
  I'm the main developer behind elasticseach-head, and I have a plee.

   I don't have a lot of time to work on this project, but I _do_ want to keep moving it forward - However, I'm not a java guy, and I'm not a search guy and I don't (anymore) work on elasticsearch day-to-day, but I am a UI guy. What I need to enable me to add features, fix bugs and feel confident about merging pull requests, is sample data. The ability to quickly spin up a cluster that has percolation, or makes extensive use of the _parent feature or has 100 indices. 

So my question: does anyone have or would be willing to contribute shell scripts (or something else) that create indices and clusters with various elasticsearch features, field types, mappings, percolation, rivers.

For example, I would love to add support for rivers to es-head, but I don't want to have to _learn_ about the intricacies of each river, it will take too much time. If there are some scripts out there already to start up a mongodb db, and start a river, then I could probably add an interface pretty quickly.

My existing sample directory https://github.com/mobz/elasticsearch-head/tree/master/lib/es/test/index_generators is, however woefully small . I would accept pull requests against this, or perhaps there is an existing resource out there. Maybe this would be a great resource for more people that could be a standalone project? Maybe kimchy has a giant stash?!

-B