Skip to main content

Upgrading to Cloudera 5 beta 2

How to upgrade Cloudera 5 beta 1 and install beta2

 To upgrade Cloudera 5 beta 1 to beta 2 there are two methods. I'm using method B. For guys who cant uninstall, they can give a shot at method A.

A. Upgrade manually ( this is from cloudera forum)
  1. Stop CDH5b1 cluster in CM5b1.
  2. Stop Cloudera Monitoring Services.
  3. Upgrade CM to 5b2.
  4. Start CM
  5. Upgrade CDH to 5b2 (for packages - manually, for parcels - using CM UI). If upgrading using parcels, DO NOT start the cluster.
  6. (Only if YARN is present in secure cluster) Goto “Settings” > “Administration”, Select ALL “yarn” principals and hit “Regenerate”. Wait for the “Generate Credentials” command to finish. This is needed because CDH5b2 added the HTTP principal for YARN.
  7. Start ZooKeeper
  8. Run HDFS Metadata Upgrade command from HDFS Actions Menu
  9. Run Oozie Database Upgrade command from Oozie Actions Menu
  10. Run Oozie Install Sharelib command from Oozie Actions Menu
  11. Run Upgrade Sqoop command from Sqoop Actions Menu
  12. Run Upgrade Hive Metastore Database Schema command from Hive Actions Menu
  13. Start all services
  14. Deploy Client Configuration for the cluster.
  15. Start Cloudera Monitoring Services.
B. Uninstall beta 1 and Install beta 2 (Recommended)

   1. Stop All cluster and stop Management
   2. Run the below script in all nodes


   sudo service cloudera-scm-agent hard_stop
   echo Y|sudo apt-get purge 'cloudera-manager-*'
   sudo apt-get clean
   sudo rm -Rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/cloudera* /var/log/cloudera* /var/run/cloudera*
   sudo rm -Rf /var/lib/flume-ng /var/lib/hadoop* /var/lib/hue /var/lib/oozie /var/lib/solr /var/lib/sqoop*
   sudo rm -Rf /dfs /mapred /yarn
   for u in hdfs mapred cloudera-scm hbase hue zookeeper oozie hive impala flume; do sudo kill $(ps -u $u -o pid=); done
   sudo rm /tmp/.scm_prepare_node.lock

  Now uninstall is complete. Download the latest install bin file from cloudera and install. 

PS: Make sure oozie is properly uninstalled. During reinstallation, creating oozie db schema created lot of issues.





Comments

Post a Comment

Popular posts from this blog

spark java.lang.IllegalArgumentException: java.net.UnknownHostException: user

Today I faced an error while trying to use Spark shell. This is how I resolved. scala> val file = sc.textFile("hdfs://...") 14/10/21 13:34:23 INFO MemoryStore: ensureFreeSpace(217085) called with curMem=0, maxMem=309225062 14/10/21 13:34:23 INFO MemoryStore: Block broadcast_0 stored as values to memory (estimated size 212.0 KB, free 294.7 MB) file: org.apache.spark.rdd.RDD[String] = MappedRDD[1] at textFile at <console>:12 scala> file.count() java.lang.IllegalArgumentException: java.net.UnknownHostException: user     at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)     at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:237)     at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:141) This error can be fixed by giving proper hostname and port sc.textFile("hdfs://{hostname}:8020/{filepath}...") scala> file.count() 14/10/21 13:44:23 INFO FileInputFormat: Tota

Common issues on Shark with CDH5-beta2

Issues on Shark with CDH5-beta2 1. IncompatibleClassChangeError: Implementing class Exception in thread "main" java.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.ClassLoader.defineC

Common Issues with Solr Data Import Handler (DIH)

1. Could not load driver: org.postgresql.Driver org.apache.solr.common.SolrException; Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load driver: org.postgresql.Driver Solution : Put rmdbs driver, in my case postgres driver in $SOLR_HOME/dist folder and point it in solrconfig.xml <lib dir="${solr.install.dir:../../../..}/dist/" regex="postgresql.*\.jar" />  2. ERROR StreamingSolrClients org.apache.solr.common.SolrException: Bad Request request: http://host:7574/solr/collection_shard2_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2Fhost%3A8983%2Fsolr%2Fcollection_shard1_replica2%2F&wt=javabin&version=2 at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:241) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurr