Friday, May 16, 2014

Exception : Error working with Sqoop 2

Solution for Sqoop2: Exception has occurred during processing command Message, returned a response status of 404 Not Found
 
This is continuous post of,  Error : Installing and working with Sqoop 2

To start Sqoop Server
root@boss[bin]#./sqoop.sh server start

Enter client mode
root@boss[bin]#./sqoop.sh client  
Sqoop home directory: /opt/software/sqoop-1.99.2-bin-hadoop100...
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000>

Configure Client
sqoop:000>  set server --host localhost --port 12000 --webapp sqoop    
Server is set successfully

Verify connnection
sqoop:000>  show version --all                                      
client version:
  Sqoop 1.99.2 revision 3e31b7d3eefb3696d4970704364dea05a9ea2a59
  Compiled by homeuser on Mon Apr 15 20:54:18 PDT 2013

Exception has occurred during processing command
Exception: com.sun.jersey.api.client.UniformInterfaceException Message: GET http://localhost:12000/sqoop/version returned a response status of 404 Not Found
sqoop:000> 


If you get above error, you may missed out sqoop installation dependencies.  

you need to install Hadoop libraries into Sqoop server. Sqoop provides convenience script addtowar.sh to do so.
[
before start the Sqoop do this, if u started Sqoop server just stop
root@boss[bin]#./sqoop.sh server stop
]

change the  Hadoop configuration directory path in sqoop.properties ($SQOOP_HOME/server/conf/) to your hadoop conf location ($HADOOP_HOME/conf)


root@boss[bin]#vi  ../server/conf/sqoop.properties 
.......
.......
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/opt/hadoop-1.1.0/conf/
.......


then add the Hadoop dependency by

root@boss[bin]#./addtowar.sh -hadoop-version 1.1 -hadoop-path /opt/hadoop-1.1.0/
Hadoop version: 1.1
Hadoop path: /opt/hadoop-1.1.0/
Extra jars:

Injecting following Hadoop JARs

/opt/hadoop-1.1.0/hadoop-core-1.1.0.jar
/opt/hadoop-1.1.0/lib/jackson-core-asl-1.8.8.jar
/opt/hadoop-1.1.0/lib/jackson-mapper-asl-1.8.8.jar
/opt/hadoop-1.1.0/lib/commons-configuration-1.6.jar
/opt/hadoop-1.1.0/lib/commons-logging-1.1.1.jar
/opt/hadoop-1.1.0/lib/slf4j-api-1.4.3.jar
/opt/hadoop-1.1.0/lib/slf4j-log4j12-1.4.3.jar

Backing up original WAR file to ./../server/webapps/sqoop.war_2014-05-16_12:33:52.818424007

New Sqoop WAR file with added 'Hadoop JARs' at ./../server/webapps/sqoop.war
  
hadoop-version : specify major version of Hadoop ex : 1.X or 2.X
hadoop-path  : specify Haoop library path usually $HADOOP_HOME path


Now re-do the same step you will get  what you expected

To start Sqoop Server

root@boss[bin]#./sqoop.sh server start

Enter client mode
root@boss[bin]#./sqoop.sh client  
Sqoop home directory: /opt/software/sqoop-1.99.2-bin-hadoop100...
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000>

Configure Client
sqoop:000>  set server --host localhost --port 12000 --webapp sqoop    
Server is set successfully

Verify connection
sqoop:000>  show version --all                                      
client version:
  Sqoop 1.99.2 revision 3e31b7d3eefb3696d4970704364dea05a9ea2a59
  Compiled by homeuser on Mon Apr 15 20:54:18 PDT 2013
server version:
  Sqoop 1.99.2 revision 3e31b7d3eefb3696d4970704364dea05a9ea2a59
  Compiled by homeuser on Mon Apr 15 20:54:18 PDT 2013
Protocol version:
  [1]
sqoop:000> 


you can also check through the browser by specified url



1 comment:

nirmala said...

Good technical post. I have found more exception and solution on sqoop in your blog. My suggestion is to provide back link to all sqoop issues.