Wednesday, January 6, 2016

RHadoop - Running Hadoop MapReduce from R

RHadoop - Running Hadoop MapReduce from R

Running Hadoop Mapreduce From R. Working with R and Hadoop for large scale data analytics

Running Mapreduce job
m <- mapreduce(InputPath, input.format = 'csv', map = mapFunction, reduce = reduceFunction )

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

Solution

Error seems map not find the file location,
I was trying to reading File from local but by backend configured with Local

rmr.options(backend = "local")

Reading File from Hadoop in R session
f = hdfs.line.reader("/HDFS/GdpData.csv");

Error in .jcheck(silent = TRUE) :
No running JVM detected. Maybe .jinit() would help.

Solution

Run .jinit() to initialize the JVM,

.jinit()

Post a Comment