ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Tony Su
Not sure why. the indexer.conf from the 10min tutorial at the following URL looks very simple and generic
 
Setup
apache.log > logstash shipper(parse apache) > redis > logstash (indexer.config) > ES 1.0 RC1
 
logstash 1.3..3
redis 2.8.4
 
Data is consumed and passed successfully to redis, then no further.
Redis and ES recognize clients like usual.
 
I have deployed using ES .0.90.10 and everything works as expected using exactly the same config files.
 
Is there a known fix? The indexer.conf is so simple I guess maybe the grok pattern is not supported?
 
Tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3d3268f4-c971-40c4-b929-70d00f3c0b2c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Binh Ly
Tony,

You should be able to get this to work as-is directly to ES 0.90.10. If you want to use another version of ES like 1.0 RC1, for now you need to change the elasticsearch output to elasticsearch_http:

  #elasticsearch {
  #  host => "localhost"
  #  port => 9300
  #}
  elasticsearch_http {
    host => "localhost"
  }

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e57881c3-9696-43e0-b73d-a325d00d1f74%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Tony Su
In reply to this post by Tony Su
Thx, I'll give that a try.
 
Since your fix merely specifies the http transport, does this mean that elasticsearch intends to support protocols other than http?
 
Thx,
Tony
 

On Thursday, January 30, 2014 2:26:42 PM UTC-8, Tony Su wrote:
Not sure why. the indexer.conf from the 10min tutorial at the following URL looks very simple and generic
<a onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Flogstash.net%2Fdocs%2F1.3.3%2Ftutorials%2F10-minute-walkthrough%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFRBYHBxRyIJ174qHA9iVALE7gq4Q';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Flogstash.net%2Fdocs%2F1.3.3%2Ftutorials%2F10-minute-walkthrough%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNFRBYHBxRyIJ174qHA9iVALE7gq4Q';return true;" href="http://logstash.net/docs/1.3.3/tutorials/10-minute-walkthrough/" target="_blank">http://logstash.net/docs/1.3.3/tutorials/10-minute-walkthrough/
 
Setup
apache.log > logstash shipper(parse apache) > redis > logstash (indexer.config) > ES 1.0 RC1
 
logstash 1.3..3
redis 2.8.4
 
Data is consumed and passed successfully to redis, then no further.
Redis and ES recognize clients like usual.
 
I have deployed using ES .0.90.10 and everything works as expected using exactly the same config files.
 
Is there a known fix? The indexer.conf is so simple I guess maybe the grok pattern is not supported?
 
Tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/89e8c31c-e915-4eea-ad1f-0fbbac6c0045%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Binh Ly
Tony, I believe Thrift and Memcached are also available in ES. But not sure if/how that would work with LS though.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-thrift.html

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-memcached.html

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8afa5de0-d6d9-45b0-9725-20e13aaefa35%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Tony Su
In reply to this post by Binh Ly
Unfortunately, adding/specifying http in the Logstash Indexer.conf still does not work communicating to ES 1.0 RC1.
 
Output follows, first the stdout error reported by Logstash followed by indexer.conf contents
 
[hidden email]> java -jar logstash-1.3.3-flatjar.jar agent -f indexer.conf
Error: Expected one of #, { at line 17, column 17 (byte 355) after output {
  stdout { debug => true debug_format => "json"}
  elasticsearch
[hidden email]> cat indexer.conf
input {
  redis {
    host => "127.0.0.1"
    # these settings should match the output of the agent
    data_type => "list"
    key => "logstash"
    # We use the 'json' codec here because we expect to read
    # json events from redis.
    codec => json
  }
}
output {
  stdout { debug => true debug_format => "json"}
  elasticsearch http{
    host => "127.0.0.1"
  }
}
 
 
 
 
 
 
 
 

On Thursday, January 30, 2014 2:41:13 PM UTC-8, Binh Ly wrote:
Tony,

You should be able to get this to work as-is directly to ES 0.90.10. If you want to use another version of ES like 1.0 RC1, for now you need to change the elasticsearch output to elasticsearch_http:

  #elasticsearch {
  #  host => "localhost"
  #  port => 9300
  #}
  elasticsearch_http {
    host => "localhost"
  }

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20eddcbe-8d8f-4fac-bdd6-033ea2ea7451%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Binh Ly
Tony, can you please confirm, the output should be "elasticsearch_http", with underscore in between.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0e4cee8e-88cd-46fd-b6bd-9d9208c312f7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: ES 1.0 RC1 - 10min tutorial "indexer.conf" broken

Tony Su
Yup.
I can't believe I didn't read that right.
Is working now.
 
Thx,
Tony
 
 

On Friday, January 31, 2014 9:36:31 AM UTC-8, Binh Ly wrote:
Tony, can you please confirm, the output should be "elasticsearch_http", with underscore in between.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a886f40a-8668-410a-8331-7cd538017fd9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.