CDHhadoop部署
1. 所有免密钥登陆(all节点)(centos6 操作系统)
创新互联专注于广元企业网站建设,自适应网站建设,商城网站制作。广元网站建设公司,为广元等地区提供建站服务。全流程按需求定制设计,专业设计,全程项目跟踪,创新互联专业和态度为您提供的服务
node01 2C 6G node02 2C 2G node03 2C 2G 2.node01 安装数据库 MySQL5.5(node01节点) https://blog.csdn.net/u012402177/article/details/82870433 root/123456 3.下载基础的软件(all 节点) yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb /etc/init.d/ntpd start 解决agent启动会有问题: /opt/cloudera-manager/cm-5.4.3/lib64/cmf/agent/build/env/bin/python: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory wget http://www.python.org/ftp/python/2.6.6/Python-2.6.6.tgz tar -xzvf Python-2.6.6.tgz cd Python-2.6.6 ./configure --prefix=/foo/python26 --enable-shared make make altinstall sudo ln -s /foo/python26/lib/libpython2.6.so.1.0 /usr/lib/libpython2.6.so.1.0 4.clouderamanager安装(server 和 agent节点都需要 all) mkdir /opt/cloudera-manager tar xvzf cloudera-manager*.tar.gz -C /opt/cloudera-manager 5.添加用户所有节点 useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 6.修该所有agent节点的server的地址 /opt/cloudera-manager/cm-5.4.3/etc/cloudera-scm-agent/config.ini中server_host 7.创建目录 Server节点 mkdir -p /opt/cloudera/parcel-repo chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo Agent节点 mkdir -p /opt/cloudera/parcels chown cloudera-scm:cloudera-scm /opt/cloudera/parcels 8.server节点制作源(文件放在如下路径) cd /opt/cloudera/parcel-repo [root@node01 parcel-repo]# 11 -rw-r--r-- 1 root root 1272629458 Oct 30 13:33 CDH-5.4.0-1.cdh6.4.0.p0.27-el6.parcel -rw-r--r-- 1 root root 41 Oct 30 13:33 CDH-5.4.0-1.cdh6.4.0.p0.27-el6.parcel.sha -rw-r--r-- 1 root root 42625 Oct 30 13:32 manifest.json 9.配置CM数据库 以及数据库连接驱动 mkdir -p /usr/share/java/ cd /usr/share/java/ cp -a /root/install/mysql-connector-java-5.1.26-bin.jar ./ mv mysql-connector-java-5.1.26-bin.jar mysql-connector-java.jar 创建库(server端执行) 删除mysql.user 里面所有多余的用户 只剩一个管理员 grant all on *.* to 'temp'@'%' identified by 'temp' with grant option; flush privileges; cd /opt/cloudera-manager/cm-5.4.3/share/cmf/schema/ ./scm_prepare_database.sh mysql temp -h node01 -utemp -ptemp --scm-host node01 scm scm scm 10.启动服务 server端 node01 cd /opt/cloudera-manager/cm-5.4.3/etc/init.d/ ./cloudera-scm-server start agent node01 node02 node03 /opt/cloudera-manager/cm-5.4.3/etc/init.d/cloudera-scm-agent restart
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
当前题目:CDHhadoop部署
本文地址:http://lswzjz.com/article/iidhos.html