离线安装Cloudera_Manager_让梦想疯狂_新浪博客
离线安装Cloudera_Manager
一：集群规划&&
/cdh5/parcels/5.6.0/​
cloudera-manager-installer.bin:&
/cm5/installer/5.6.0/
cloudera-managerr
/cm5/redhat/7/x86_64/cm/5.6.0/RPMS/x86_64
​​
二：集群准备
1：配置网络(部分省略)
2：修改主机名&
/etc/hostname&(部分省略)
3：添加ip与主机名的映射&&
/etc/hosts
&(部分省略)
4:关闭防火墙(可以只执行第二步：推荐)
1）永久关闭防火墙
/etc/selinux/config，将其中的SELinux设置：SELINUX=disabled
2)&关闭防火墙
systemctldisable firewalld.service
3)&关闭packagekit:
&/etc/yum/pluginconf.d/langpacks.conf，将enabled设为0
5:配置ssh&
(建议root与普通用户都执行)&&
(部分省略)
1:所有节点生成ssh密钥对, ssh-keygen
2:进入~/.ssh文件，将每个机器生成的公钥id_rsa.pub的内容复制到本机的
~/.ssh/&的authorized_keys&文件
里。命令如下：
cpid_rsa.pub authorized_keys
3:修改权限
chmod700 ~/.ssh
chmod600 ~/.ssh/authorized_keys
sshlocalhost，如果可以免密码登录，则为成功
5:把slave1的公钥拷贝到其它节点，实现slave1到slave[1~5]的免密码登录ssh-copy-id
slave（依次到2~5）
6：slave2到slave1的ssh免密码登录ssh-copy-idslave1
Server节点（slave1）
1：安装rpm包
把需要的安装包拷贝到&&
/home/hadoop/app目录下
yumlocalinstall
&-nogpgcheck&&*.rpm(安装rpm包)
2：执行./cloudera-manager-installer.bin，如果遇到问题修改一下即可
agent节点（slave2~5）
1：把下面的包分发到agent节点上
/opt/hadoop/app&目录下
cloudera-manager-agent-5.6.0-1.cm560.p0.54.el7.x86_64.rpm
cloudera-manager-daemons-5.6.0-1.cm560.p0.54.el7.x86_64.rpm
2:agent安装
yumlocalinstall
&-nogpgcheck&&*.rpm(安装rpm包)
3:启动服务
servicecloudera-scm-server start
servicecloudera-scm-agent start
4:打开浏览器就可以看到安装页面
Http://ip:7181
初始用户与密码admin
Server节点（slave1）配置本地库
/etc/cloudera-scm-agent/config.ini
修改前为：localhost&
&修改后为：localhost.localdomain
2：把以下载的.parcel文件和manifest.json文件拷贝到/opt/cloudera/parcel-repo/文件夹下
3：把CDH-5.6.0-1.cdh5.6.0.p0.45-el7.parcel.sha1命名为CDH-5.6.01.cdh5.6.0.p0.45-el7.parcel.sha
4：打开manifest.json文件查看CDH-5.6.0-1.cdh5.6.0.p0.45-el7.parcel对应的编号，并拷贝到CDH-5.6.0-1.cdh5.6.0.p0.45-el7.parcel.sha文件中
​以下是manifest.json中CDH-5.6.0-1.cdh5.6.0.p0.45-el7.parcel对应的编号，这里是：
e67e08e2ee1f74e6c15
"parcelName": "CDH-5.6.0-1.cdh5.6.0.p0.45-el7.parcel",
"components": [
& "pkg_version": "0.7.0+cdh5.6.0+0",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "bigtop-tomcat",
& "version": "6.0.44-cdh5.6.0"
& "pkg_version": "0.11.0+cdh5.6.0+77",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "crunch",
& "version": "0.11.0-cdh5.6.0"
& "pkg_version": "1.6.0+cdh5.6.0+30",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "flume-ng",
& "version": "1.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-0.20-mapreduce",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-hdfs",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-httpfs",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-kms",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-mapreduce",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "2.6.0+cdh5.6.0+1023",
& "pkg_release": "1.cdh5.6.0.p0.120",
& "name": "hadoop-yarn",
& "version": "2.6.0-cdh5.6.0"
& "pkg_version": "1.0.0+cdh5.6.0+300",
& "pkg_release": "1.cdh5.6.0.p0.109",
& "name": "hbase",
& "version": "1.0.0-cdh5.6.0"
& "pkg_version": "1.5+cdh5.6.0+57",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "hbase-solr",
& "version": "1.5-cdh5.6.0"
& "pkg_version": "1.1.0+cdh5.6.0+377",
& "pkg_release": "1.cdh5.6.0.p0.110",
& "name": "hive",
& "version": "1.1.0-cdh5.6.0"
& "pkg_version": "1.1.0+cdh5.6.0+377",
& "pkg_release": "1.cdh5.6.0.p0.110",
& "name": "hive-hcatalog",
& "version": "1.1.0-cdh5.6.0"
& "pkg_version": "3.9.0+cdh5.6.0+365",
& "pkg_release": "1.cdh5.6.0.p0.105",
& "name": "hue",
& "version": "3.9.0-cdh5.6.0"
& "pkg_version": "2.4.0+cdh5.6.0+0",
& "pkg_release": "1.cdh5.6.0.p0.112",
& "name": "impala",
& "version": "2.4.0-cdh5.6.0"
& "pkg_version": "1.0.0+cdh5.6.0+119",
& "pkg_release": "1.cdh5.6.0.p0.100",
& "name": "kite",
& "version": "1.0.0-cdh5.6.0"
& "pkg_version": "1.0.0+cdh5.6.0+0",
& "pkg_release": "1.cdh5.6.0.p0.100",
& "name": "llama",
& "version": "1.0.0-cdh5.6.0"
& "pkg_version": "0.9+cdh5.6.0+26",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "mahout",
& "version": "0.9-cdh5.6.0"
& "pkg_version": "4.1.0+cdh5.6.0+235",
& "pkg_release": "1.cdh5.6.0.p0.108",
& "name": "oozie",
& "version": "4.1.0-cdh5.6.0"
& "pkg_version": "1.5.0+cdh5.6.0+174",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "parquet",
& "version": "1.5.0-cdh5.6.0"
& "pkg_version": "0.12.0+cdh5.6.0+73",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "pig",
& "version": "0.12.0-cdh5.6.0"
& "pkg_version": "1.5.1+cdh5.6.0+120",
& "pkg_release": "1.cdh5.6.0.p0.105",
& "name": "sentry",
& "version": "1.5.1-cdh5.6.0"
& "pkg_version": "4.10.3+cdh5.6.0+339",
& "pkg_release": "1.cdh5.6.0.p0.105",
& "name": "solr",
& "version": "4.10.3-cdh5.6.0"
& "pkg_version": "1.5.0+cdh5.6.0+113",
& "pkg_release": "1.cdh5.6.0.p0.104",
& "name": "spark",
& "version": "1.5.0-cdh5.6.0"
& "pkg_version": "1.99.5+cdh5.6.0+34",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "sqoop2",
& "version": "1.99.5-cdh5.6.0"
& "pkg_version": "1.4.6+cdh5.6.0+33",
& "pkg_release": "1.cdh5.6.0.p0.101",
& "name": "sqoop",
& "version": "1.4.6-cdh5.6.0"
& "pkg_version": "0.9.0+cdh5.6.0+17",
& "pkg_release": "1.cdh5.6.0.p0.100",
& "name": "whirr",
& "version": "0.9.0-cdh5.6.0"
& "pkg_version": "3.4.5+cdh5.6.0+91",
& "pkg_release": "1.cdh5.6.0.p0.99",
& "name": "zookeeper",
& "version": "3.4.5-cdh5.6.0"
"replaces": "IMPALA, SOLR, SPARK",
& & & "hash":
"e67e08e2ee1f74e6c15"
现在本地源已配好，总之parcel-repo文件中包含一下三个文件：
如果安装失败，或者想要卸载，请参考
/thread-.html
安装参考：
/thread-.html
让梦想疯狂
博客等级：
博客积分：0
博客访问：3,346
关注人气：0
荣誉徽章：KIKI王 的BLOG
用户名：KIKI王
文章数：33
访问量：3671
注册日期：
阅读量：5863
阅读量：12276
阅读量：384250
阅读量：1075566
51CTO推荐博文
前提：使用root用户登录系统，减少不必要的麻烦，以及用户切换（推荐几个工具）----------------1.PieTTY 0.3.26.exe：window下安全的远程连接程序（window下连接虚拟机中linux）。2.Winscp：支持ssh的文件传输软件。（可以在window下实现和linux文件交互）1.在虚拟机下安装好linux系统2.设置静态ip，右键工作栏上电脑符号，修改ip。 &2.1重启网卡：service network restart &2.2验证：ifconfig，查看是否正确，ping，查看是否和物理机联通3.修改主机名： &3.1修改当前会话主机名：hostname 主机名 &3.2修改配置文件主机名：vi /etc/sysconfig/network中 & & hostname=主机名4.把主机名和ip绑定 &4.1vi /etc/hosts 增加一行 & & & & & & & & & & &192.168.80.100 主机名（hadoop） & & 验证：在物理机上 ping hadoop5.关闭防火墙运行（一定要关闭） &service iptables stop &验证：service iptables status6.关闭防火墙自动运行 &执行命令：chkconfig iptables off7.SSH免登陆（具体见 博客：ssh免登陆） &7.1切换到所在目录 &/root/.ssh文件夹 &7.2执行命令产生秘（公）钥
& & & & & &ssh-keygen -t rsa & & 会产生两个文件 id_rsa.pub和id_rsa. &7.3执行命令 cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys7.4验证： ssh 主机名 看是否需要密码8.安装jdk（我们的jdk和hadoop都安装在/usr/local下）执行命令 删除所有内容使用工具把文件从复制到下执行命令 赋予执行权限 文件名进行解压缩 文件名要在下e) 执行命令重命名 &f) 设置环境变量
& 增加两行内容 & export JAVA_HOME=/usr/local/jdkexport PATH=.:$JAVA_HOME/bin:$PATH保存退出设置立即生效：验证：8.安装执行命令 进行解压缩重命名 设置环境变量 增加修改增加执行修改配置文件修改四个配置文件 hadoop-env.sh、、、hadoop-env.sh 修改第九行 core-site.xml具体修改------------------------------------------con/core-site.xml:&configuration&&property& & &&name&fs.default.name&/name& &--#是描述hadoop集群中NameNode节点的url，集群中其他的节点必须知道NameNode的位置，DataNode必须先在NameNoe上注册，客户端可以通过这个url与NamNode进行数据交互--& & &&value&hdfs://hadoop:9000&/value& & &&description&change your own hostname&/description&&/property&&property& & &&name&hadoop.tmp.dir&/name& &--hadoop.tmp.dir是hadoop文件系统依赖的基础配置，NameNode和DataNode的默认存放位置--& & &&value&/usr/local/hadoop/tmp&/value&&/property&&/configuration&conf/mapred-site.xml:&configuration&&property& & &&name&mapred.job.tracker&/name& &&--jobtracker所在的主机和端口，--& & &&value&hadoop:9001&/value&&/property&&/configuration&conf/hdfs-site.xml:&configuration&&property& & &&name&dfs.replication&/name&&--这个配置决定了hdfs中有保留多少个备份，默认是3个，没有上限，但是过多的备份会占用资源，太少又不安全--& & &&value&1&/value&&/property&&property& & &&name&dfs.permissions&/name& & &&value&false&/value&&/property&&/configuration&-----------------------------------------------对进行格式化： & &hadoop namenode -format执行命令启动执行命令验证，发现个进程通过浏览器查看（）在下查看，修改文件192.168.80.100 hadoop然后可以直接 &hadoop:50070进行查看
了这篇文章
类别：┆阅读(0)┆评论(0)使用Cloudera Manager安装Hadoop - 推酷
使用Cloudera Manager安装Hadoop
Hadoop是由好多不同的服务（比如HDFS，Hive，HBase，Spark等等）构成的，这些服务之间还有些依赖关系，如果直接下Apache上的原始的包，需要下载多次，配置多次，显得比较麻烦。由此就产生了一些对Hadoop进行定制的公司，比如
. 这些公司都有自己的Hadoop发行版，Cloudera的发行版叫CDH。因为公司的系统用的就是CDH，这一段时间装了多次CDH5，做了一些实际操作，还看了些Cloudera的文档。
Cloudera使用Cloudera Manager，一个Web界面的Hadoop管理系统来进行Hadoop相关服务的安装，配置，和监控。
安装Cloudera Manager
官方指南在
，有几种方式，我这里说到的只是其中之一，不是最简单的，但是可以让你对Cloudera Manager有比较多的了解。
可以通过下面这张简单的图来大概了解下Cloudera Manager是如何去管理Hadoop服务的。
配置Cloudera Manager的YUM源
在/etc/yum.repos.d/下加一个文件CM.repo,内容如下.
baseurl=http://archive-/cm5/redhat/6/x86_64/cm/5.1.3/
gpgcheck=0
baseurl = http : //archive-/cm5/redhat/6/x86_64/cm/5.1.3/
gpgcheck = 0
运行下面的命令查看Yum仓库配置成功与否，如果yum list能列出cloudera-manager-***那些包，就说明成功了。如果你是在实验室环境里安装，可能无法连接到外网，那么你可以在一台可以连接外网的机器上用reposync命令把外网的yum源clone到你本地，然后本地起个http服务给实验室的机器当yum服务器。也可以在实验室的CM.repo文件配置代理来连接外网，具体步骤Google一下就行了。
[root@bogon yum.repos.d]# yum clean all
Loaded plugins: fastestmirror, refresh-packagekit, security
Cleaning repos: CM base extras updates
Cleaning up Everything
Cleaning up list of fastest mirrors
[root@bogon yum.repos.d]# yum list | grep cloudera
cloudera-manager-daemons.x86_64
5.1.3-1.cm513.p0.155.el6
cloudera-manager-server.x86_64
5.1.3-1.cm513.p0.155.el6
cloudera-manager-agent.x86_64
5.1.3-1.cm513.p0.155.el6
cloudera-manager-server-db-2.x86_64
5.1.3-1.cm513.p0.155.el6
[ root @ bogon yum . repos . d ] # yum clean all
Loaded plugins : fastestmirror , refresh - packagekit , security
Cleaning repos : CM base extras updates
Cleaning up Everything
Cleaning up list of fastest mirrors
[ root @ bogon yum . repos . d ] # yum list | grep cloudera
cloudera - manager - daemons . x86_64 5.1.3 - 1.cm513.p0.155.el6 @ CM
cloudera - manager - server . x86_64 5.1.3 - 1.cm513.p0.155.el6 @ CM
cloudera - manager - agent . x86_64 5.1.3 - 1.cm513.p0.155.el6 CM
cloudera - manager - server - db - 2.x86_64 5.1.3 - 1.cm513.p0.155.el6 CM
安装Cloudera Manager相关的包, 安装这个包时会自动把它依赖的cloudera-manager-daemons包也装上。
yum install cloudera-manager-server
yum install cloudera - manager - server
看看这两个包到底装些什么,可以看到server基本上就复制了一些配置文件.真正的管理网站是通过daemons装的，具体的文件都在/usr/share/cmf目录下，日志在/var/log/cloudera-scm-server目录下，而运行时的配置文件在/var/run/cloudera-scm-server
[root@bogon ~]# rpm -ql cloudera-manager-server
/etc/cloudera-scm-server
/etc/cloudera-scm-server/db.properties
/etc/cloudera-scm-server/log4j.properties
/etc/default/cloudera-scm-server
/etc/rc.d/init.d/cloudera-scm-server
/etc/security/limits.d/cloudera-scm.conf
/opt/cloudera/csd
/opt/cloudera/parcel-repo
/usr/sbin/cmf-server
/var/lib/cloudera-scm-server
[root@bogon ~]# rpm -ql cloudera-manager-daemons
/usr/share/cmf/yarn-fixtures/details_1.json
/usr/share/cmf/yarn-fixtures/details_2.json
/usr/share/cmf/yarn-fixtures/details_3.json
/usr/share/cmf/yarn-fixtures/details_4.json
/usr/share/cmf/yarn-fixtures/details_5.json
/usr/share/cmf/yarn-fixtures/details_6.json
/usr/share/cmf/yarn-fixtures/details_7.json
/usr/share/cmf/yarn-fixtures/details_8.json
/usr/share/cmf/yarn-fixtures/details_9.json
/var/log/cloudera-scm-server
/var/run/cloudera-scm-server
[ root @ bogon ~ ] # rpm -ql cloudera-manager-server
/ etc / cloudera - scm - server
/ etc / cloudera - scm - server / db . properties
/ etc / cloudera - scm - server / log4j . properties
/ etc / default / cloudera - scm - server
/ etc / rc . d / init . d / cloudera - scm - server
/ etc / security / limits . d / cloudera - scm . conf
/ opt / cloudera / csd
/ opt / cloudera / parcel - repo
/ usr / sbin / cmf - server
/ var / lib / cloudera - scm - server
[ root @ bogon ~ ] # rpm -ql cloudera-manager-daemons
/ usr / share / cmf / yarn - fixtures / details_1 . json
/ usr / share / cmf / yarn - fixtures / details_2 . json
/ usr / share / cmf / yarn - fixtures / details_3 . json
/ usr / share / cmf / yarn - fixtures / details_4 . json
/ usr / share / cmf / yarn - fixtures / details_5 . json
/ usr / share / cmf / yarn - fixtures / details_6 . json
/ usr / share / cmf / yarn - fixtures / details_7 . json
/ usr / share / cmf / yarn - fixtures / details_8 . json
/ usr / share / cmf / yarn - fixtures / details_9 . json
/ var / log / cloudera - scm - server
/ var / run / cloudera - scm - server
安装数据库和对应的driver，这是因为Cloudera Manager本身需要把配置保存数据库中，另外后面装Hive时也需要数据库来保存元数据。
yum install mysql-server
yum install mysql-connector-java.noarch
yum install mysql - server
yum install mysql - connector - java . noarch
接着配置一些数据库相关的参数，先在数据库添加数据库cmf，以及对应的用户cmf，还有密码。
mysql& create database cmf DEFAULT CHARACTER SET utf8;
Query OK, 1 row affected (0.00 sec)
mysql& grant all on cmf.* TO 'cmf'@'localhost' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)
mysql & create database cmf DEFAULT CHARACTER SET utf8 ;
Query OK , 1 row affected ( 0.00 sec )
mysql & grant all on cmf . * TO 'cmf' @ 'localhost' IDENTIFIED BY '123456' ;
Query OK , 0 rows affected ( 0.00 sec )
然后修改 /etc/cloudera-scm-server/db.properties文件，配置Cloudera manager要访问哪个数据库，对应的用户名，密码等。
# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com.cloudera.cmf.db.type=mysql
# The database host
# If a non standard port is needed, use 'hostname:port'
com.cloudera.cmf.db.host=localhost
# The database name
com.cloudera.cmf.db.name=cmf
# The database user&a href=&http://linmingren.me/blog/wp-content/uploads/2014/09/cloudera-welcome.png&&&img src=&http://linmingren.me/blog/wp-content/uploads/2014/09/cloudera-welcome-.png& alt=&& title=&cloudera welcome& width=&584& height=&213& class=&alignnone size-large wp-image-479& /&&/a&
com.cloudera.cmf.db.user=cmf
# The database user's password
com.cloudera.cmf.db.password=123456
# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com . cloudera . cmf . db . type = mysql
# The database host
# If a non standard port is needed, use 'hostname:port'
com . cloudera . cmf . db . host = localhost
# The database name
com . cloudera . cmf . db . name = cmf
# The database user&a href=&http://linmingren.me/blog/wp-content/uploads/2014/09/cloudera-welcome.png&&&img src=&http://linmingren.me/blog/wp-content/uploads/2014/09/cloudera-welcome-.png& alt=&& title=&cloudera welcome& width=&584& height=&213& class=&alignnone size-large wp-image-479& /&&/a&
com . cloudera . cmf . db . user = cmf
# The database user's password
com . cloudera . cmf . db . password = 123456
现在就可以启动cloudera-scm-server服务了, 用netstat命令看下7180端口有没有启动（从服务启动到端口开启在我的机器上大概要半分钟），有的话就可以在浏览器中输入localhost:7180来访问Cloudera Manger的web界面了 （默认用户密码是admin/admin），没有的话请看/var/log/cloudera-scm-server/cloudera-scm-server.log中的异常是什么。
service cloudera-scm-server restart
service cloudera - scm - server restart
第一次登陆进去的首页是这
已发表评论数()
请填写推刊名
描述不能大于100个字符!
权限设置： 公开
仅自己可见
正文不准确
标题不准确
排版有问题
主题不准确
没有分页内容
图片无法显示
视频无法显示
与原文不一致