Correct way to use TransportClient connection object

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Correct way to use TransportClient connection object

bagui
Hi,

 I'm using the below code to get a singleton object for TransportClient object. I'm using the getInstance() to get the client object which is already alive in webapplication. 

public static Client getInstance()
{
if (instance == null)
 {
logger.debug("the client instance is null, creating a new instance");
ImmutableSettings.Builder settings = ImmutableSettings.settingsBuilder(); 
       settings.put("node.client", true); 
       settings.put("node.data", false); 
       settings.put("node.name", "node-client");
       settings.put("cluster.name", "elasticsearch");
       settings.build(); 
instance = new TransportClient(settings)
               .addTransportAddress(new InetSocketTransportAddress("10.203.238.139", 9300));
logger.debug("returning the new created client instance...");
return instance;
 }
return instance;
}

Calling the client as below from search api.
Client client = ESClientFactory.getInstance(); 

Now the issue is if I don't close client like client.close() I'm getting memory leak warning from webserver tomcat side. If I do close the connection using client.close() after search api call then I'm getting NoNodeAvailableException exception.

Please suggest what is the correct way to call the connection object.

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2aaf77f-cd18-4e52-98fc-c25ed03601fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

joergprante@gmail.com
Closing the transport client may not be enough.

Try this:

- wait for all outstanding actions (all actions send responses asynchronously)
- then shut down client.threadpool() (perhaps with shutdownNow() or shutdown()), this effectively disables new actions form being started
- then close the transport client with client.close()
- then set the client instance to null to allow GC to clean up

Also you should put the static transport client object in a ServetContextListener so the web app container is able to manage start and stop in contextInitialized() and contextDestroyed()

Beside this, memory leak warnings from tomcat are warnings. They may be right or wrong.

Jörg


On Fri, Jun 6, 2014 at 7:43 AM, Subhadip Bagui <[hidden email]> wrote:
Hi,

 I'm using the below code to get a singleton object for TransportClient object. I'm using the getInstance() to get the client object which is already alive in webapplication. 

public static Client getInstance()
{
if (instance == null)
 {
logger.debug("the client instance is null, creating a new instance");
ImmutableSettings.Builder settings = ImmutableSettings.settingsBuilder(); 
       settings.put("node.client", true); 
       settings.put("node.data", false); 
       settings.put("node.name", "node-client");
       settings.put("cluster.name", "elasticsearch");
       settings.build(); 
instance = new TransportClient(settings)
               .addTransportAddress(new InetSocketTransportAddress("10.203.238.139", 9300));
logger.debug("returning the new created client instance...");
return instance;
 }
return instance;
}

Calling the client as below from search api.
Client client = ESClientFactory.getInstance(); 

Now the issue is if I don't close client like client.close() I'm getting memory leak warning from webserver tomcat side. If I do close the connection using client.close() after search api call then I'm getting NoNodeAvailableException exception.

Please suggest what is the correct way to call the connection object.

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2aaf77f-cd18-4e52-98fc-c25ed03601fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHL3rAeBsHLrWYkm4-MqO7oU_9CJmL7Ge6SPS15FoyJmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

bagui
In reply to this post by bagui
Hi Jorg,

Sorry to open the thread again. But the issue is I'm getting OOM error currently in tomcat and the webapplication is crushing. As u suggested I'm calling the static TransportClient instance from contextInitialized() and shutting down with contextDestroyed(). But the methods are not getting called it seems. Trying like below.
Can you please check.

public class ESClientFactory implements ServletContextListener {

    /** The logger. */
    private static Logger logger = Logger.getLogger(ESClientFactory.class);
    
    /** The instance. */
    public static TransportClient instance;

    /**
     * Instantiates a new eS client factory.
     */
    private ESClientFactory() {
    }

    /**
     * Gets the single instance of ESClientFactory.
     *
     * @return single instance of ESClientFactory
     */
    public static Client getInstance() {
        String ipAddress = MessageTranslator.getMessage("es.cluster.ip");
        int transportClientPort = 0;
        String clusterName = MessageTranslator.getMessage("es.cluster.name");
        
        try {
            transportClientPort =
                Integer.parseInt(MessageTranslator
                    .getMessage("es.transportclient.port"));
        }
        catch (Exception e) {
            transportClientPort = 9300;
            LogImpl.setWarning(ESClientFactory.class, e);
        }

        logger.debug("got the client ip as :" + ipAddress + " and port :"
            + transportClientPort);
        if (instance == null) {
            logger
                .debug("the client instance is null, creating a new instance");
            ImmutableSettings.Builder settings =
                ImmutableSettings.settingsBuilder();
            settings.put("cluster.name", clusterName);
            settings.build();
            instance =
                new TransportClient(settings)
                    .addTransportAddress(new InetSocketTransportAddress(
                        ipAddress, transportClientPort));
            
            logger.debug("returning the new created client instance...");
            return instance;
        }
        logger
            .debug("returning the existing transport client object connection.");
        return instance;
    }


@Override
public void contextInitialized(ServletContextEvent sce) {
logger.debug("initializing the servletContextListener... TransportClient");
getInstance();
}

@Override
public void contextDestroyed(ServletContextEvent sce) {
logger.debug("closing the servlet context");
shutdown();
logger.debug("successfully shutdown threadpool");
}

public synchronized void shutdown() {
        if (instance != null) {
            logger.debug("shutdown started");
            instance.close();
            instance.threadPool().shutdown();
            instance = null;
            logger.debug("shutdown complete");
        }
    }

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/814fdb41-461b-4b5c-8bad-835039d76c24%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

Ed Kim
Did you declare the listener in your web.xml?

On Tuesday, January 13, 2015 at 3:27:15 AM UTC-8, Subhadip Bagui wrote:
Hi Jorg,

Sorry to open the thread again. But the issue is I'm getting OOM error currently in tomcat and the webapplication is crushing. As u suggested I'm calling the static TransportClient instance from contextInitialized() and shutting down with contextDestroyed(). But the methods are not getting called it seems. Trying like below.
Can you please check.

public class ESClientFactory implements ServletContextListener {

    /** The logger. */
    private static Logger logger = Logger.getLogger(ESClientFactory.class);
    
    /** The instance. */
    public static TransportClient instance;

    /**
     * Instantiates a new eS client factory.
     */
    private ESClientFactory() {
    }

    /**
     * Gets the single instance of ESClientFactory.
     *
     * @return single instance of ESClientFactory
     */
    public static Client getInstance() {
        String ipAddress = MessageTranslator.getMessage("es.cluster.ip");
        int transportClientPort = 0;
        String clusterName = MessageTranslator.getMessage("<a href="http://es.cluster.name" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fes.cluster.name\46sa\75D\46sntz\0751\46usg\75AFQjCNEeTmtu5bpkAPxzB3zxr1tvDXt8rA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fes.cluster.name\46sa\75D\46sntz\0751\46usg\75AFQjCNEeTmtu5bpkAPxzB3zxr1tvDXt8rA';return true;">es.cluster.name");
        
        try {
            transportClientPort =
                Integer.parseInt(MessageTranslator
                    .getMessage("es.transportclient.port"));
        }
        catch (Exception e) {
            transportClientPort = 9300;
            LogImpl.setWarning(ESClientFactory.class, e);
        }

        logger.debug("got the client ip as :" + ipAddress + " and port :"
            + transportClientPort);
        if (instance == null) {
            logger
                .debug("the client instance is null, creating a new instance");
            ImmutableSettings.Builder settings =
                ImmutableSettings.settingsBuilder();
            settings.put("<a href="http://cluster.name" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcluster.name\46sa\75D\46sntz\0751\46usg\75AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fcluster.name\46sa\75D\46sntz\0751\46usg\75AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g';return true;">cluster.name", clusterName);
            settings.build();
            instance =
                new TransportClient(settings)
                    .addTransportAddress(new InetSocketTransportAddress(
                        ipAddress, transportClientPort));
            
            logger.debug("returning the new created client instance...");
            return instance;
        }
        logger
            .debug("returning the existing transport client object connection.");
        return instance;
    }


@Override
public void contextInitialized(ServletContextEvent sce) {
logger.debug("initializing the servletContextListener... TransportClient");
getInstance();
}

@Override
public void contextDestroyed(ServletContextEvent sce) {
logger.debug("closing the servlet context");
shutdown();
logger.debug("successfully shutdown threadpool");
}

public synchronized void shutdown() {
        if (instance != null) {
            logger.debug("shutdown started");
            instance.close();
            instance.threadPool().shutdown();
            instance = null;
            logger.debug("shutdown complete");
        }
    }

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fae4cd13-9f9f-4187-9e23-93d8c73c3132%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

bagui
In reply to this post by bagui
Yeah. I'm using listener, but it was giving some error. Fixed it and now it's working. My mistake :$

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/26d40ffe-4d0b-4c49-a81c-e86e5f091b9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

joergprante@gmail.com
In Servlet 3.0 you can also add @WebServletContextListener as annotation to the class, without having to use web.xml 

Example:


On Wed, Jan 14, 2015 at 7:50 AM, Subhadip Bagui <[hidden email]> wrote:
Yeah. I'm using listener, but it was giving some error. Fixed it and now it's working. My mistake :$

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/26d40ffe-4d0b-4c49-a81c-e86e5f091b9a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGhcP72aYs9arcM_VCqu2HZDp3e%2B2w5sf5NhXPE8DMqbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

bagui
In reply to this post by bagui
Hi,

In the same context... some times when I'm shutting down tomcat getting the below exception. And other times it works. Any idea why ?

Jan 19, 2015 8:59:30 AM org.apache.catalina.core.StandardContext listenerStop
SEVERE: Exception sending context destroyed event to listener instance of class com.aricent.aricloud.es.service.ESClientFactory
java.lang.NoClassDefFoundError: org/elasticsearch/transport/netty/NettyTransport$4
at org.elasticsearch.transport.netty.NettyTransport.doStop(NettyTransport.java:403)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at org.elasticsearch.transport.TransportService.doStop(TransportService.java:100)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at org.elasticsearch.common.component.AbstractLifecycleComponent.close(AbstractLifecycleComponent.java:117)
at org.elasticsearch.client.transport.TransportClient.close(TransportClient.java:268)
at com.aricent.aricloud.es.service.ESClientFactory.shutdown(ESClientFactory.java:118)
at com.aricent.aricloud.es.service.ESClientFactory.contextDestroyed(ESClientFactory.java:111)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4b948428-c260-4aef-ad82-93346e7488cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Correct way to use TransportClient connection object

joergprante@gmail.com
This is an exception because TransportClient still has open requests and was not closed in time, i.e. before Tomcat closed the web app and removed the class loader.

Jörg

On Mon, Jan 19, 2015 at 4:37 AM, Subhadip Bagui <[hidden email]> wrote:
Hi,

In the same context... some times when I'm shutting down tomcat getting the below exception. And other times it works. Any idea why ?

Jan 19, 2015 8:59:30 AM org.apache.catalina.core.StandardContext listenerStop
SEVERE: Exception sending context destroyed event to listener instance of class com.aricent.aricloud.es.service.ESClientFactory
java.lang.NoClassDefFoundError: org/elasticsearch/transport/netty/NettyTransport$4
at org.elasticsearch.transport.netty.NettyTransport.doStop(NettyTransport.java:403)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at org.elasticsearch.transport.TransportService.doStop(TransportService.java:100)
at org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at org.elasticsearch.common.component.AbstractLifecycleComponent.close(AbstractLifecycleComponent.java:117)
at org.elasticsearch.client.transport.TransportClient.close(TransportClient.java:268)
at com.aricent.aricloud.es.service.ESClientFactory.shutdown(ESClientFactory.java:118)
at com.aricent.aricloud.es.service.ESClientFactory.contextDestroyed(ESClientFactory.java:111)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4b948428-c260-4aef-ad82-93346e7488cb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFv4vXGUHK%3DGR11UDb6KZ1N8jT7WMGeKkpj1UCcSwV45w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.