> thank you so much for your quick reply,
> Here is what I had done
> 1-instaled hadoop-1.2.1( pig-0.12.0 / hive-0.11.0 /...)
> 2-download Elasticsearch-1.0.1 and put it in the same file of hadoop
> 3-copied the following 4 elasticsearch-hadoop jar files
> elasticsearch-hadoop-1.3.0.M2.jar
> elasticsearch-hadoop-1.3.0.M2-sources.jar
> elasticsearch-hadoop-1.3.0.M2-javadoc.jar
> elasticsearch-hadoop-1.3.0.M2-yarn.jar
> to /pig and hadoop/lib
> 4- Add them in the PIG_CLASSPATH
>
> knowing that when I take data from my Desktop and put it in elasticsearch using pig script it works very well, but when
> I try to get data from my HDFS it gives me that :
>
> 2014-05-12 23:16:31,765 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR: java.io.IOException: Out of
> nodes and retries; caught exception
> 2014-05-12 23:16:31,765 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2014-05-12 23:16:31,766 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt Features
> 1.2.1 0.12.0 hduser 2014-05-12 23:15:34 2014-05-12 23:16:31 GROUP_BY
>
> Failed!
>
> Failed Jobs:
> JobId Alias Feature Message Outputs
> job_201405122310_0001 weblog_count,weblog_group,weblogs GROUP_BY,COMBINER Message: Job failed! Error - # of
> failed Reduce Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201405122310_0001_r_000000
> weblogs1/logs2,
>
> Input(s):
> Failed to read data from "/user/weblogs"
>
> Output(s):
> Failed to produce result in "weblogs1/logs2"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> job_201405122310_0001
>
>
> 2014-05-12 23:16:31,766 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
> Failed!
>
>
> And here is the script :
>
> weblogs = LOAD '/user/weblogs' USING PigStorage('\t')
> AS (client_ip : chararray,
> full_request_date : chararray,
> day : int,
> month : chararray,
> month_num : int,
> year : int,
> hour : int,
> minute : int,
> second : int,
> timezone : chararray,
> http_verb : chararray,
> uri : chararray,
> http_status_code : chararray,
> bytes_returned : chararray,
> referrer : chararray,
> user_agent : chararray
> );
> weblog_group = GROUP weblogs by (client_ip, year, month_num);
> weblog_count = FOREACH weblog_group GENERATE group.client_ip, group.year, group.month_num, COUNT_STAR(weblogs) as pageviews;
> STORE weblog_count INTO 'weblogs1/logs2' USING org.elasticsearch.hadoop.pig.EsStorage();
>
>
> Le lundi 12 mai 2014 16:28:20 UTC+1, Costin Leau a écrit :
>
> Check your network settings and make sure that the Hadoop nodes can communicate with the ES nodes.
> If you install ES besides Hadoop itself, this shouldn't be a problem.
> There are various way to check this - try ping, tracert, etc...
>
> Please refer to your distro manual/documentation for more information about the configuration and setup.
>
> Cheers,
>
> On 5/12/14 3:42 PM, hanine haninne wrote:
> > I had get the same erreur but I don't know what I have to change in my "/etc/hosts"
> > thank you for your help
> >
> > Le mercredi 5 mars 2014 09:39:46 UTC, Yann Barraud a écrit :
> >
> > Hi,
> >
> > Is your ES instance known by your Hadoop cluster (/etc/hosts) ?
> >
> > It does not even seems to read in it.
> >
> > Cheers,
> > Yann
> >
> > Le mercredi 5 mars 2014 06:32:55 UTC+1, siva mannem a écrit :
> >
> > I installed ES(at the location /usr/lib/elasticsearch/) on our gateway server and i am able to run some basic
> > curl commands like XPUT and XGET to create some indices and retrieve the data in them.
> > i am able to give single line JSON record but i am unable to give JSON file as input to curl XPUT .
> > can anybody give me the syntax for giving JSON file as input for curl XPUT command?
> >
> > my next issue is i copied the following 4 elasticsearch-hadoop jar files
> > elasticsearch-hadoop-1.3.0.M2.jar
> > elasticsearch-hadoop-1.3.0.M2-sources.jar
> > elasticsearch-hadoop-1.3.0.M2-javadoc.jar
> > elasticsearch-hadoop-1.3.0.M2-yarn.jar
> >
> > to /usr/lib/elasticsearch/elasticsearch-0.90.9/lib
> > and /usr/lib/gphd/pig/
> >
> > i have the following json file j.json
> > ++++++
> > {"k1":"v1" , "k2":"v2" , "k3":"v3"}
> > ++++++++
> >
> > in my_hdfs_path.
> >
> > my pig script is write_data_to_es.pig
> > +++++++++++++
> > REGISTER /usr/lib/gphd/pig/elasticsearch-hadoop-1.3.0.M2-yarn.jar;
> > DEFINE ESTOR org.elasticsearch.hadoop.pig.EsStorage('es.resource=usa/ca');
> > A = LOAD '/my_hdfs_path/j.json' using JsonLoader('k1:chararray,k2:chararray,k3:chararray');
> > STORE A into 'usa/ca' USING ESTOR('es.input.json=true');
> > ++++++++++++++
> >
> > when i run my pig script
> > +++++++++
> > pig -x mapreduce write_data_to_es.pig
> > ++++++++++++
> >
> > i am getting following error
> > +++++++++
> > Input(s):
> > Failed to read data from "/my_hdfs_path/j.json"
> >
> > Output(s):
> > Failed to produce result in "usa/ca"
> >
> > Counters:
> > Total records written : 0
> > Total bytes written : 0
> > Spillable Memory Manager spill count : 0
> > Total bags proactively spilled: 0
> > Total records proactively spilled: 0
> >
> > Job DAG:
> > job_1390436301987_0089
> >
> >
> > 2014-03-05 00:26:50,839 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
> > 2014-03-05 00:26:50,841 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Input(s):
> > Failed to read data from "/elastic_search/es_hadoop_test.json"
> >
> > Output(s):
> > Failed to produce result in "mannem/siva"
> >
> > Counters:
> > Total records written : 0
> > Total bytes written : 0
> > Spillable Memory Manager spill count : 0
> > Total bags proactively spilled: 0
> > Total records proactively spilled: 0
> >
> > Job DAG:
> > job_1390436301987_0089
> >
> > 2014-03-05 00:26:50,839 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
> > 2014-03-05 00:26:50,841 [main] ERROR org.apache.pig.tools.grunt.GruntParser - *ERROR 2997: Encountered
> > IOException. Out of nodes and retries; caught exception*
> > Details at logfile: /usr/lib/elasticsearch/elasticsearch-0.90.9/pig_1393997175206.log
> > ++++++++++++
> >
> > i am using pivotal hadoop version (1.0.1) which is basically apache hadoop (hadoop-2.0.2)
> > and pig version is 0.10.1
> > and elastic search version is 0.90.9
> >
> > can anybody help me out here?
> > thank you so much in advance for your help.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to
> >
[hidden email] <javascript:> <mailto:
[hidden email] <javascript:>>.
> > To view this discussion on the web visit
> >
https://groups.google.com/d/msgid/elasticsearch/1dd8ff7d-ef53-4614-9300-13b5f6ed66fa%40googlegroups.com> <
https://groups.google.com/d/msgid/elasticsearch/1dd8ff7d-ef53-4614-9300-13b5f6ed66fa%40googlegroups.com>
> > <
https://groups.google.com/d/msgid/elasticsearch/1dd8ff7d-ef53-4614-9300-13b5f6ed66fa%40googlegroups.com?utm_medium=email&utm_source=footer> <
https://groups.google.com/d/msgid/elasticsearch/1dd8ff7d-ef53-4614-9300-13b5f6ed66fa%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visithttps://groups.google.com/d/optout <
https://groups.google.com/d/optout>.
>
> --
> Costin
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>
[hidden email] <mailto:
[hidden email]>.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/elasticsearch/cd9d3143-556a-43c8-9cfd-78b666db48b7%40googlegroups.com> <
https://groups.google.com/d/msgid/elasticsearch/cd9d3143-556a-43c8-9cfd-78b666db48b7%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit
https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
Locked
