Skip to main content

Upgrading to Cloudera 5 beta 2

How to upgrade Cloudera 5 beta 1 and install beta2

 To upgrade Cloudera 5 beta 1 to beta 2 there are two methods. I'm using method B. For guys who cant uninstall, they can give a shot at method A.

A. Upgrade manually ( this is from cloudera forum)
  1. Stop CDH5b1 cluster in CM5b1.
  2. Stop Cloudera Monitoring Services.
  3. Upgrade CM to 5b2.
  4. Start CM
  5. Upgrade CDH to 5b2 (for packages - manually, for parcels - using CM UI). If upgrading using parcels, DO NOT start the cluster.
  6. (Only if YARN is present in secure cluster) Goto “Settings” > “Administration”, Select ALL “yarn” principals and hit “Regenerate”. Wait for the “Generate Credentials” command to finish. This is needed because CDH5b2 added the HTTP principal for YARN.
  7. Start ZooKeeper
  8. Run HDFS Metadata Upgrade command from HDFS Actions Menu
  9. Run Oozie Database Upgrade command from Oozie Actions Menu
  10. Run Oozie Install Sharelib command from Oozie Actions Menu
  11. Run Upgrade Sqoop command from Sqoop Actions Menu
  12. Run Upgrade Hive Metastore Database Schema command from Hive Actions Menu
  13. Start all services
  14. Deploy Client Configuration for the cluster.
  15. Start Cloudera Monitoring Services.
B. Uninstall beta 1 and Install beta 2 (Recommended)

   1. Stop All cluster and stop Management
   2. Run the below script in all nodes


   sudo service cloudera-scm-agent hard_stop
   echo Y|sudo apt-get purge 'cloudera-manager-*'
   sudo apt-get clean
   sudo rm -Rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/cloudera* /var/log/cloudera* /var/run/cloudera*
   sudo rm -Rf /var/lib/flume-ng /var/lib/hadoop* /var/lib/hue /var/lib/oozie /var/lib/solr /var/lib/sqoop*
   sudo rm -Rf /dfs /mapred /yarn
   for u in hdfs mapred cloudera-scm hbase hue zookeeper oozie hive impala flume; do sudo kill $(ps -u $u -o pid=); done
   sudo rm /tmp/.scm_prepare_node.lock

  Now uninstall is complete. Download the latest install bin file from cloudera and install. 

PS: Make sure oozie is properly uninstalled. During reinstallation, creating oozie db schema created lot of issues.





Comments

Post a Comment

Popular posts from this blog

Upgrading nodejs in ubuntu 14.04

My machine has 5.x installed and had lot of trouble updating it to 8.x. Below are the steps I followed to upgrade nodejs from 5.x to 8.x #add the new source list sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 68576280  sudo apt-add-repository "deb https://deb.nodesource.com/node_8.x $(lsb_release -sc) main" sudo apt-get update #Remove the previous installation sudo apt-get purge nodejs npm  #Verify if proper version is going to be installed apt-cache policy <package> #Install new version sudo apt-get install -y nodejs

Common issues on Shark with CDH5-beta2

Issues on Shark with CDH5-beta2 1. IncompatibleClassChangeError: Implementing class Exception in thread "main" java.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.ClassLoader.defineC...

org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

Recently installed the latest cloudera hadoop. First issue I faced while working with hive. Diagnostic Messages for this Task: Container launch failed for container_1406173012885_0009_01_000021 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container . This token is expired. current time is 1406254943000 found 1406254938244     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)     at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:152)     at org.apache.hadoop.yarn.api.records.impl.pb.Serial...