Friday, April 17, 2015

1 step to Move the file from remote system to Hadoop HDFS

1 step to Move the file from remote system to Hadoop HDFS
1 command to move the file from remote system to Hadoop HDFS cluster.

In some case we would need to move the file from non installed
Hadoop system to Hadoop cluster.

move file from Non Hadoop system to Hadoop cluster

    bdalab@solai:/opt$ cat moveToHdfs.txt | ssh userName@NameNodeIP "hadoop dfs -put - hadoopDirName/"


here,
moveToHdfs.txt -> is a file to be moved to HDFS
userName -> userName of NameNode system
NameNodeIP -> IP address of NameNode
hadoopDirName -> Dir in HDFS


If you face any error like

    bash: hadoop: command not found
re-run the above command by hadoop full path

    bdalab@solai:/opt$ cat moveToHdfs.txt | ssh userName@NameNodeIP "/opt/hadoop-2.6.0/bin/hadoop dfs -put - hadoopDirName/"
Post a Comment