Hadoop job log4j:WARN Please initialize the log4j system properly

[neustre]

'm trying to make a mapreduce job in Rstudio in order to make some statistics , and i' having this error , searched every where , i didn't find the reason : so here is my code in Rstudio:

Sys.setenv(HADOOP_CMD="/usr/local/hadoop/bin/hadoop")
Sys.setenv(HADOOP_HOME="/usr/local/hadoop")
Sys.setenv(JAVA_HOME="/usr/local/java/jre")
Sys.setenv(HADOOP_CONF="/usr/local/hadoop/conf")
Sys.setenv(HADOOP_STREAMING="/usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar")
library("rmr2")
library("rhdfs")
set <- ("/data.csv")
mapreduce(set,
map=function(k,v) keyval(k,v),
input.format = "csv",
output.format = "native")

but i get this :

Error in mr(mp=map,reduce=reduce,combine=combine,vectorized.reduce, :
hadoop streaming failed with error code 1

in the log File i see this :

log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Try to solve this for 4 days . Any HELP would be greatly appreciated !!!

[piccolbo]

Please simplify your example by removing unnecessary rhdfs load, try to read a sample of data with read.table to verify format and provide stderr of failing task (accessible through mapreduce web UI).

[neustre]

hi , i did verify again all the logs , and i saw the functional package problem , after searching , i discover that the problem is that i wasn't root . once i logged in as root and reinstall the packages again, i've made the countword example and it works .
I have another question not really a question but i just want to know the process if i can reach you in private . thank you .

[piccolbo]

For private support one option is to contact Revolution Analytics, they
offer a variety of solution.

On Thu, Jul 2, 2015 at 12:17 AM neustre [email protected] wrote:

hi , i did verify again all the logs , and i saw the functional package
problem , after searching , i discover that the problem is that i wasn't
root . once i logged in as root and reinstall the packages again, i've made
the countword example and it works .
I have another question not really a question but i just want to know the
process if i can reach you in private . thank you .

—
Reply to this email directly or view it on GitHub
#170 (comment)
.

[neustre]

ok, i only have a project to do with hadoop and R i wanted to know steps , and if i am on the good road , project has a deadline . i'll contact them any way , thank you .

[neustre]

hi , i have a question please , i didn't find an answer. Can i do a shiny application based on rhdfs and rmr .
if i finished my tests and algorithms with mapreduce and R , can i be able to do shiny without any problem or there is a comparability issue ?

Thank you