HDFS setup issues

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

HDFS setup issues

Praveen Kariyanahalli
Installed hadoop plugin. But I now see that elastic search is not bound to port 9200. Any idea why? Before I added gateway commands, all was working fine, before I changed gateway from local to hdfs. 

Here is the conf I added.

===== elasticsearch.yml

gateway.type: hdfs
gateway.hdfs.uri: hdfs://127.0.0.1:54311
gateway.hdfs.path: /app/hadoop/tmp

==========

NOTE: mapred-site is pointing to 54311. Also all elasticsearch/hadoop all running on a single m/c 


====netstat o/p

praveen@praveen-laptop:~/CodeJul/milkyway/sol/server/scripts/api$ sudo netstat -tapun | grep LISTEN
tcp        0      0 0.0.0.0:35619           0.0.0.0:*               LISTEN      4023/java       
tcp        0      0 127.0.0.1:54310         0.0.0.0:*               LISTEN      4023/java       
tcp        0      0 127.0.0.1:54311         0.0.0.0:*               LISTEN      4197/java       
tcp        0      0 0.0.0.0:4743            0.0.0.0:*               LISTEN      1094/openhpid   
tcp        0      0 0.0.0.0:59080           0.0.0.0:*               LISTEN      4700/java       
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      4700/java       
tcp        0      0 127.0.0.1:34510         0.0.0.0:*               LISTEN      4930/java       
tcp        0      0 0.0.0.0:50030           0.0.0.0:*               LISTEN      4197/java       
tcp        0      0 0.0.0.0:28017           0.0.0.0:*               LISTEN      2328/mongod     
tcp        0      0 127.0.0.1:61619         0.0.0.0:*               LISTEN      1224/python2.6  
tcp        0      0 0.0.0.0:61620           0.0.0.0:*               LISTEN      1224/python2.6  
tcp        0      0 0.0.0.0:40598           0.0.0.0:*               LISTEN      4197/java       
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      4023/java       

DONOT see 9200 atall (or any port around it)

=====

$ ps -eaf | grep elast
root     10003     1  1 15:14 pts/2    00:00:08 /usr/bin/java -Xms256m -Xmx256m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=/home/praveen/es/elasticsearch-0.19.4 -cp :/home/praveen/es/elasticsearch-0.19.4/lib/*:/home/praveen/es/elasticsearch-0.19.4/lib/sigar/* -Des.max-open-files=true -Des.index.storage.type=memory org.elasticsearch.bootstrap.ElasticSearch

Any help is greatly appreciated !

Thanks
-pk

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: HDFS setup issues

Praveen Kariyanahalli
Can somebody share their working sample yml file/ map-site xml file ? That might give me some hints.

Thanks in Advance
-pk

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: HDFS setup issues

Praveen Kariyanahalli
It is working now.

fs.default.name was pointing to 54311 and not 54310.

The port number in the 
gateway.hdfs.uri should point to namenode port (defined in the core-site.xml). My bad.

  <name>fs.default.name</name>  <value>hdfs://localhost:54310</value>


On Friday, August 10, 2012 5:24:08 PM UTC-7, Praveen Kariyanahalli wrote:
Can somebody share their working sample yml file/ map-site xml file ? That might give me some hints.

Thanks in Advance
-pk

--
 
 
Reply | Threaded
Open this post in threaded view
|

Re: HDFS setup issues

Praveen Kariyanahalli
In reply to this post by Praveen Kariyanahalli
It is working now!

The port in the gateway.hdfs.uri was wrongly pointing to 54311 instead of 54310. 

Actually it should point to filesystem port number. which is defined in the core-site.xml

  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>

Regards
-pk

On Friday, August 10, 2012 5:24:08 PM UTC-7, Praveen Kariyanahalli wrote:
Can somebody share their working sample yml file/ map-site xml file ? That might give me some hints.

Thanks in Advance
-pk

--