In the latest version of CDH5.2, while trying to run spark-shell will encounter this error.
Solution:
utils.sh can be downloaded from github. I'm not sure this is the perfect solution, but things seems to be working after putting the file
1. get the file from
https://github.com/apache/spark/blob/master/bin/utils.sh
2. copy utils.sh to /opt/cloudera/parcels/CDH/lib/spark/bin/
user@spark-master:~# spark-shell
/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/bin/../lib/spark/bin/spark-shell: line 44: /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/bin/utils.sh: No such file or directory
Solution:
utils.sh can be downloaded from github. I'm not sure this is the perfect solution, but things seems to be working after putting the file
1. get the file from
https://github.com/apache/spark/blob/master/bin/utils.sh
2. copy utils.sh to /opt/cloudera/parcels/CDH/lib/spark/bin/
user@spark-master:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/bin# spark-shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/jars/spark-assembly-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/10/16 11:05:22 INFO SecurityManager: Changing view acls to: user
14/10/16 11:05:22 INFO SecurityManager: Changing modify acls to: user
14/10/16 11:05:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(user); users with modify permissions: Set(user)
14/10/16 11:05:22 INFO HttpServer: Starting HTTP Server
14/10/16 11:05:22 INFO Utils: Successfully started service 'HTTP class server' on port 50045.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.1.0
Comments
Post a Comment