ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Paolo Castagna
Hi,
are the ElasticSearch artifacts published to any Maven/Ivy repository?

What are the jars needed to use the ElasticSearch Java APIs?

Paolo
Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

kimchy
Administrator
No, not yet. I am trying to get it published to central, as usual, its a pain. The jar files it needs are basically the ones listed under the lib directory.

-shay.banon

On Tue, Mar 30, 2010 at 3:17 PM, Paolo Castagna <[hidden email]> wrote:
Hi,
are the ElasticSearch artifacts published to any Maven/Ivy repository?

What are the jars needed to use the ElasticSearch Java APIs?

Paolo

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Paolo Castagna
Shay Banon wrote:
> No, not yet. I am trying to get it published to central, as usual,
 > its a pain.

It would be nice for all the ElasticSearch users to be able to say:

   <dependency>
     <groupId>com.elasticsearch</groupId>
     <artifactId>elasticsearch</artifactId>
     <version>0.5.1</version>
   </dependency>

or:

   <dependency org="com.elasticsearch" name="elasticsearch" rev="0.5.1"/>

... and have Maven or Ivy (or something else :-)) resolve and manage
the dependencies for them.

I have never used Gradle, but it does not seem to be too painful.
Perhaps the pains are coming from somewhere else. :-)

Is there something I can do to help on this?

> The jar files it needs are basically the ones listed under the lib
> directory.

Are all necessary even if I just want to use the Java client APIs?

Thanks,
Paolo
Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

kimchy
Administrator
On Tue, Mar 30, 2010 at 7:42 PM, Paolo Castagna <[hidden email]> wrote:
Shay Banon wrote:
No, not yet. I am trying to get it published to central, as usual,
> its a pain.

It would be nice for all the ElasticSearch users to be able to say:

 <dependency>
   <groupId>com.elasticsearch</groupId>
   <artifactId>elasticsearch</artifactId>
   <version>0.5.1</version>
 </dependency>

or:

 <dependency org="com.elasticsearch" name="elasticsearch" rev="0.5.1"/>

... and have Maven or Ivy (or something else :-)) resolve and manage
the dependencies for them.

Publishing to maven central is more complex than just creating the deliverables (which do need munging when using gradle). The above is what it means to publish to an online repository.
 

I have never used Gradle, but it does not seem to be too painful.
Perhaps the pains are coming from somewhere else. :-)

As usual with maven, its coming from most/all places. Gradle does poses challenges to create proper distributions, but then just getting it to central is another pain.
 

Is there something I can do to help on this?

Not in this case. Its not technical problems.
 


The jar files it needs are basically the ones listed under the lib directory.

Are all necessary even if I just want to use the Java client APIs?

If you use the server client, then yes. Is there a problem with it?
 

Thanks,
Paolo

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Paolo Castagna
Shay Banon wrote:
> Is there something I can do to help on this?
>
> Not in this case. Its not technical problems.

Also, I think one issue might be that there are dependencies from
another repository (i.e. http://repository.jboss.com/maven2/).

  "All the dependencies have to already be present in the
   central repository."
   --
http://maven.apache.org/guides/mini/guide-central-repository-upload.html

There are alternatives though:

  - Have your own repo
  - Use third parties, for example:
    https://docs.sonatype.com/display/NX/OSS+Repository+Hosting

>     Are all necessary even if I just want to use the Java client APIs?
>
> If you use the server client, then yes. Is there a problem with it?

There isn't a specific problem with it, but I like to understand the
dependencies and keep them to the minimum number, strictly necessary.

It's one way to make the Jar hell problem, a bit less hell. :-)

Thanks again,
Paolo
Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

kimchy
Administrator
All the jar files under the lib directory are required if you are using a cluster of elasticsearch instances.

-shay.banon

On Tue, Mar 30, 2010 at 11:40 PM, Paolo Castagna <[hidden email]> wrote:
Shay Banon wrote:
Is there something I can do to help on this?

Not in this case. Its not technical problems.

Also, I think one issue might be that there are dependencies from
another repository (i.e. http://repository.jboss.com/maven2/).

 "All the dependencies have to already be present in the
 central repository."
 -- http://maven.apache.org/guides/mini/guide-central-repository-upload.html

There are alternatives though:

 - Have your own repo
 - Use third parties, for example:
  https://docs.sonatype.com/display/NX/OSS+Repository+Hosting


   Are all necessary even if I just want to use the Java client APIs?

If you use the server client, then yes. Is there a problem with it?

There isn't a specific problem with it, but I like to understand the dependencies and keep them to the minimum number, strictly necessary.

It's one way to make the Jar hell problem, a bit less hell. :-)

Thanks again,
Paolo

Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Lukáš Vlček
It seems to me that Paolo is asking what JARs are needed if the Java program has to talk to elasticsearch in no-cluster-joining mode, i.e. TransportClient. One nasty way how to learn this would be to do a simple test and keep adding one JAR to classpath at a time from ES/lib folder until it stops complaining about ClassNotFoundException :-)

Lukas

On Tue, Mar 30, 2010 at 10:50 PM, Shay Banon <[hidden email]> wrote:
All the jar files under the lib directory are required if you are using a cluster of elasticsearch instances.

-shay.banon


On Tue, Mar 30, 2010 at 11:40 PM, Paolo Castagna <[hidden email]> wrote:
Shay Banon wrote:
Is there something I can do to help on this?

Not in this case. Its not technical problems.

Also, I think one issue might be that there are dependencies from
another repository (i.e. http://repository.jboss.com/maven2/).

 "All the dependencies have to already be present in the
 central repository."
 -- http://maven.apache.org/guides/mini/guide-central-repository-upload.html

There are alternatives though:

 - Have your own repo
 - Use third parties, for example:
  https://docs.sonatype.com/display/NX/OSS+Repository+Hosting


   Are all necessary even if I just want to use the Java client APIs?

If you use the server client, then yes. Is there a problem with it?

There isn't a specific problem with it, but I like to understand the dependencies and keep them to the minimum number, strictly necessary.

It's one way to make the Jar hell problem, a bit less hell. :-)

Thanks again,
Paolo


Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

Paolo Castagna
Lukáš Vlček wrote:
> It seems to me that Paolo is asking what JARs are needed if the Java
> program has to talk to elasticsearch in no-cluster-joining mode, i.e.
> TransportClient. One nasty way how to learn this would be to do a simple
> test and keep adding one JAR to classpath at a time from ES/lib folder
> until it stops complaining about ClassNotFoundException :-)

Indeed it would be good to know both for the Server Client (for which
Shay is telling us it's all) and for the Transport Client. Is the
Transport Client using an HTTP client and using the REST APIs?

Yes, the 'nasty' way you describe is what I do sometimes, but it is a
method which does not give me complete confidence since you might not
touch code with your simple test which requires jars loaded later at
runtime.

My initial question, though, was about the Server Client.

I'll try and test the Transport Client as well. Indeed, I would like
some advice on what are the pros/cons in term of performances and
data transfer between the client and the nodes in the ElasticSearch
clusters.

The documentation says (about the Transport Client): "most actions will
probably be 'two hop' operations". I inferred from this that the
Server Client is better than the Transport Client. (I'll have a look
at the sources, to better understand the differences).

Paolo



Reply | Threaded
Open this post in threaded view
|

Re: ElasticSearch artifacts to use via Ant+Ivy and/or Maven

kimchy
Administrator
On Wed, Mar 31, 2010 at 10:44 AM, Paolo Castagna <[hidden email]> wrote:
Lukáš Vlček wrote:
It seems to me that Paolo is asking what JARs are needed if the Java program has to talk to elasticsearch in no-cluster-joining mode, i.e. TransportClient. One nasty way how to learn this would be to do a simple test and keep adding one JAR to classpath at a time from ES/lib folder until it stops complaining about ClassNotFoundException :-)

Indeed it would be good to know both for the Server Client (for which
Shay is telling us it's all) and for the Transport Client. Is the
Transport Client using an HTTP client and using the REST APIs?

The Transport Client still uses the low level transport module (which is TCP based, with custom serialization/deserialization). I might, in the future, implement a REST client also (which can implement the same Client interface), but it requires some work (parsing back the JSON into the domain driven response objects).
 

Yes, the 'nasty' way you describe is what I do sometimes, but it is a
method which does not give me complete confidence since you might not
touch code with your simple test which requires jars loaded later at
runtime.

Yes, thats why its 'nasty' :). I answered regarding the TransportClient on another thread, which is, it should not require all the jars (for example, the jgroups discovery is not needed). But, I have not tested it with a subset of jars.
 

My initial question, though, was about the Server Client.

I'll try and test the Transport Client as well. Indeed, I would like
some advice on what are the pros/cons in term of performances and
data transfer between the client and the nodes in the ElasticSearch
clusters.

The documentation says (about the Transport Client): "most actions will
probably be 'two hop' operations". I inferred from this that the
Server Client is better than the Transport Client. (I'll have a look
at the sources, to better understand the differences).

The ServerClient will perform better. I will give an example. When you want to index a document, the server client will do the routing on the client side, and "hit" the shard that will handle the indexing. The TransportClient on the other hand simply does a round robin of requests to different nodes, and probably, that node will need another hop to hit the actual shard that will handle the indexing.
 

Paolo