Elasticsearch inserting date type documents as UTC timezone datetime while indexing

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
Hi,

I'm using below code to insert some documents in elasticsearch index. But when inserting in the es the time is coming as UTC format rather than original GMT+5:30Z format. Where as the Sysout before indexing is giving me correct format as  2014-08-11T18:23:13.447+05:30 . Please let me know how to keep the orizinal time format.

public static IndexResponse addDocumentsQuota(Client client, String index,
String type, String categoryName) {

IndexResponse indexResponse = null;
DateTime date = new DateTime();
System.out.println(date);
try {
indexResponse = client.prepareIndex(index, type)
.setSource(jsonBuilder()
   .startObject()
   .field("@timestamp", date)
   .field("category_name", categoryName)
   .field("alert_message", "alert message")
   .field("creation_time", date)
   .endObject())
   .execute()
   .actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return indexResponse;
}

   After indexing:

"_source": {
               "@timestamp": "2014-08-11T11:57:59.839Z",
               "category_name": "newCat1",
               "alert_message": "alert message",
               "creation_time": "2014-08-11T11:57:59.852Z"
            }

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c82c9ab-a627-46a9-ad18-dda334524ab1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
Hi,

Any ideas how to prevent the time changing while indexing in es, or to convert in correct format while query ?

Thanks,
Subhadip 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/33363b16-b2df-4bef-aded-db198c13d5ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
In reply to this post by bagui
Hi,

Can someone please give me a hint, I'm having trouble getting a solution for this.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d42cb94b-681d-4bd3-bf21-d955cd0af729%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
In reply to this post by bagui
Hi,

Please help with suggestion.



On Monday, August 11, 2014 7:51:00 PM UTC+5:30, Subhadip Bagui wrote:
Hi,

I'm using below code to insert some documents in elasticsearch index. But when inserting in the es the time is coming as UTC format rather than original GMT+5:30Z format. Where as the Sysout before indexing is giving me correct format as  2014-08-11T18:23:13.447+05:30 . Please let me know how to keep the orizinal time format.

public static IndexResponse addDocumentsQuota(Client client, String index,
String type, String categoryName) {

IndexResponse indexResponse = null;
DateTime date = new DateTime();
System.out.println(date);
try {
indexResponse = client.prepareIndex(index, type)
.setSource(jsonBuilder()
   .startObject()
   .field("@timestamp", date)
   .field("category_name", categoryName)
   .field("alert_message", "alert message")
   .field("creation_time", date)
   .endObject())
   .execute()
   .actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return indexResponse;
}

   After indexing:

"_source": {
               "@timestamp": "2014-08-11T11:57:59.839Z",
               "category_name": "newCat1",
               "alert_message": "alert message",
               "creation_time": "2014-08-11T11:57:59.852Z"
            }

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a0a623d6-524c-4928-a35d-d042855c1834%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
In reply to this post by bagui
Hi,

Please help with suggestions. 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02d96b0a-99a7-43f3-b245-13418dece674%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

Sumit Gupta
This post has NOT been accepted by the mailing list yet.
Try to insert using Milliseconds.

Thanks,
Sumit
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

Masaru Hasegawa
In reply to this post by bagui
HI,

XContentBuilder (I assume jsonBuilder() returns it) serialises date using UTC timezone by default.
If you’d like to use different format, you’d need to build your own DateTimeFormatter and pass it when you add date type field.

For example, to use ISO date time format with specified timezone:
--------
DateTimeFormatter formatter = ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone(ZONE_NAME)));
    :
.field("@timestamp", date, formatter)
    :
.field("creation_time", date, formatter)
    :
--------


Masaru

On January 5, 2015 at 13:45:19, Subhadip Bagui ([hidden email]) wrote:

> Hi,
>  
> Please help with suggestions.
>  
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch"  
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].  
> To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02d96b0a-99a7-43f3-b245-13418dece674%40googlegroups.com.  
> For more options, visit https://groups.google.com/d/optout.
>  

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54ab8619.6b8b4567.13b%40citra.local.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

bagui
In reply to this post by bagui
Hi,

I tried with what you suggested using below code. But getting exception. 

DateTime dateTime = new org.joda.time.DateTime();
System.out.println(dateTime);
DateTimeFormatter dateTimeFormatter = ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone("Asia/Calcutta")));

try {
response = client
.prepareIndex("node-test", "node-entry")
.setSource(XContentFactory.jsonBuilder()
.startObject()
.field("NodeName", nodeName)
.field("NodeStatus", nodeStatus)
.field("@timestamp", dateTime, dateTimeFormatter)
.endObject())
.execute().actionGet();
} catch (ElasticsearchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream
at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab0ae36-cd4e-42cc-bf8b-8e59ed24e678%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
laltu
Software Engineer
Reply | Threaded
Open this post in threaded view
|

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

Masaru Hasegawa
Hi,

Not sure what’s failing because of deserialize issue. It usually happens when you use different version of java.
You can see what the problem is by checking ES log instead too.

To check you are sending document in correct format, you may want to print JSON before sending index request:
--------------
XContentBuilder builder = XContentFactory.jsonBuilder().startObject()
         :
    .field("@timestamp", dateTime, dateTimeFormatter)
    .endObject();

System.out.println(builder.string());

try{
    client.prepareIndex(“node-test”, “node-entry”).setSource(builder).execute().actionGet();
} catch (Exception e) {
    e.printStackTrace();
}
--------------


Masaru

On January 7, 2015 at 20:12:59, Subhadip Bagui ([hidden email]) wrote:

> Hi,
>  
> I tried with what you suggested using below code. But getting exception.
>  
> DateTime dateTime = new org.joda.time.DateTime();
> System.out.println(dateTime);
> DateTimeFormatter dateTimeFormatter =
> ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone("Asia/Calcutta")));  
>  
> try {
> response = client
> .prepareIndex("node-test", "node-entry")
> .setSource(XContentFactory.jsonBuilder()
> .startObject()
> .field("NodeName", nodeName)
> .field("NodeStatus", nodeStatus)
> .field("@timestamp", dateTime, dateTimeFormatter)
> .endObject())
> .execute().actionGet();
> } catch (ElasticsearchException e) {
> // TODO Auto-generated catch block
> e.printStackTrace();
> }
>  
> org.elasticsearch.transport.TransportSerializationException: Failed to
> deserialize exception response from stream
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)  
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)  
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)  
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)  
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)  
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)  
>  
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch"  
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].  
> To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eab0ae36-cd4e-42cc-bf8b-8e59ed24e678%40googlegroups.com.  
> For more options, visit https://groups.google.com/d/optout.
>  

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.54ae2239.74b0dc51.677%40citra.local.
For more options, visit https://groups.google.com/d/optout.