CDH集群安装与Hadoop组件部署
吴脂娟 2022-03-18

• CDH集群安装
  • 环境准备
  • CDH部署
• Hadoop组件部署

CDH集群安装

环境准备

1. 准备5台服务器

IP	主机名	CPU	内存	硬盘
192.168.4.100	cluster0	12核	64G	15.8T
192.168.4.101	cluster1	12核	64G	15.8T
192.168.4.102	cluster2	12核	64G	15.8T
192.168.4.103	cluster3	12核	64G	15.8T
192.168.4.100	cluster4	12核	64G	15.8T

2. 修改主机名：

# 所有节点
echo "NETWORKING=yes">> /etc/sysconfig/network
echo "NETWORKING_IPV6=no">> /etc/sysconfig/network
echo "HOSTNAME=cluster0">> /etc/sysconfig/network

echo "192.168.4.100 cluster0">> /etc/hosts
echo "192.168.4.101 cluster1">> /etc/hosts
echo "192.168.4.102 cluster2">> /etc/hosts
echo "192.168.4.103 cluster3">> /etc/hosts
echo "192.168.4.104 cluster4">> /etc/hosts

3. 关闭所有节点的selinux,修改sysctl.conf

sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config
echo "vm.swappiness = 10" >> /etc/sysctl.conf
sysctl -p
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local
echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >> /etc/rc.local

4. 关闭所有节点的防火墙和清空防火墙规则链

systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld
iptables -F

5. 重启服务器 reboot
6. 设置所有节点的时区一致及时钟同步

# 所有节点
timedatectl set-timezone Asia/Shanghai
yum install -y ntp
# 主节点cluster1
vi /etc/ntp.conf
server 127.127.1.0 iburst local clock 
restrict 192.168.4.1 mask 255.255.255.0 nomodify notrap
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org
server 127.127.1.0
Fudge 127.127.1.0 stratum 10

systemctl start ntpd
systemctl enable ntpd
systemctl status ntpd
ntpq -p
# 其他节点
systemctl start ntpd
systemctl enable ntpd
/usr/sbin/ntpdate cluster0
crontab -e
00 00 * * * /usr/sbin/ntpdate cluster0

6. 主机安装插件httpd并设置开机启动

 yum install -y httpd
 systemctl start httpd
 systemctl enable httpd

7. 主机设置免密登入其他节点

 cd /root
 ssh-keygen -t rsa
 ssh-copy-id cluster0
 ssh-copy-id cluster1
 ssh-copy-id cluster2
 ssh-copy-id cluster3
 ssh-copy-id cluster4

8. 所有节点安装相关插件

 yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs redhat-lsb screen telnet lrzsz krb5-workstation krb5-libs python-pip psycopg2 /lib/lsb/init-functions mod_ssl openssl-devel python-psycopg2 Mysql-python fuse

9. 扩容

sudo tar cvf /run/home.tar /home # 备份/home分区内容
sudo fuser -km /home # 终止所有使用/home文件系统的进程,不要在/home目录下执行
sudo umount /home # 卸载
sudo lvremove /dev/mapper/centos-home # 删除/home所在的逻辑卷lv,选择y
sudo lvextend -L +2T /dev/mapper/centos-root # 扩大根目录所在的逻辑卷，这里增大1.6T
sudo xfs_growfs /dev/mapper/centos-root # 扩大/文件系统
sudo lvcreate -L 9.7T -n /dev/mapper/centos-home # 重建/home文件系统所需要的逻辑卷
sudo mkfs.xfs  /dev/mapper/centos-home # 创建文件系统
sudo mount /dev/mapper/centos-home # 将新建的文件系统挂载到/home目录下
sudo tar xvf /run/home.tar -C /  # 恢复/home目录的内容
sudo rm -rf /run/home.tar  # 删除/run下面的备份


## 添加新磁盘 /dev/sdb 对已有目录扩容
pvcreate /dev/sdb  # 创建逻辑卷
pvdisplay # 查看下
lvdisplay # 查看lv的信息
vgdisplay # 首先查看下扩容前的vg信息
vgextend  centos  /dev/sdb # 扩容VG
vgdisplay # 查看VG的信息
lvextend -L +2T /dev/mapper/centos-root # 扩容LV:
xfs_growfs /dev/mapper/centos-root  # 重置逻辑卷
lvextend -L +8.8T /dev/mapper/centos-home # 扩容LV:
xfs_growfs /dev/mapper/centos-home  # 重置逻辑卷

CDH部署

1. 下载安装包，并上传到各节点的opt目录下

• 主节点：
  jdk-8u181-linux-x64.tar.gz
  mysql-connector-java-5.1.34-bin.jar
  cdh:
  CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
  CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
  CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
  manifest.json
  clouderaManager:
  cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
  cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
  cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
  cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
  enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
  phoenix:
  PHOENIX-1.0.jar
  PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel
  PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha
  manifest.json
• 其他节点：
  scp xxxx clusterx:/opt
  jdk-8u181-linux-x64.tar.gz
  mysql-connector-java-5.1.34-bin.jar
  cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
  cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm

2. 每个节点部署JDK【Java的路径必须放在/usr/java下】

cd /root
mkdir /usr/java
tar -xzvf jdk-8u181-linux-x64.tar.gz
mv jdk1.8.0_181 /usr/java/

vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_181
export PATH=/usr/java/jdk1.8.0_181/bin:$PATH

source /etc/profile
which java

3. 主节点cluster0部署离线parcel源

mv /opt/cdh /var/www/html/
mv /opt/phoenix /var/www/html/cdh/phoenix
# 备份节点1与主节点系统

4. 主节点部署cm server与agent

cd /opt/clouderaManager
rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
cp /var/www/html/cdh/phoenix/PHOENIX-1.0.jar /opt/cloudera/csd

5. 主节点安装Mariadb

cd /opt
#安装
yum -y install mariadb mariadb-server
#启动和开机自启
systemctl start mariadb && systemctl enable mariadb
#配置Mariadb数据库
mysql_secure_installation 
#首先是设置密码，会提示先输入密码.。
Enter current password for root (enter for none):<–初次运行直接回车
#设置密码
Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车
New password: root
Re-enter new password: admin@2022
#其他配置
Remove anonymous users? [Y/n] <– 是否删除匿名用户，回车
Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,
Remove test database and access to it? [Y/n] <– 是否删除test数据库，回车
Reload privilege tables now? [Y/n] <– 是否重新加载权限表，回车

6. 所有节点配置MySQL JDBC

cd /opt
mkdir -p /usr/share/java/
cp mysql-connector-java-5.1.34-bin.jar /usr/share/java/mysql-connector-java.jar

7. 主节点配置数据库，创建CDH相关表，用户

mysql -uroot -padmin@2022

# 创建相关新库
# Cloudera Manager Server:
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
grant all privileges on scm.* to 'scm'@'cluster0' identified by 'scm@123' with grant option;

# Activity Monitor:
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon@123';
grant all privileges on amon.* to 'amon'@'cluster0' identified by 'amon@123' with grant option;

# Reports Manager:
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
grant all privileges on rman.* to 'rman'@'cluster0' identified by 'rman@123' with grant option;

# Hue:
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
grant all privileges on hue.* to 'hue'@'cluster0' identified by 'hue@123' with grant option;

# Hive Metastore Server:
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
grant all privileges on metastore.* to 'metastore'@'cluster0' identified by 'metastore@123' with grant option;

# Sentry Server:
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
grant all privileges on sentry.* to 'sentry'@'cluster0' identified by 'sentry@123' with grant option;

# Cloudera Navigator Audit Server:
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
grant all privileges on nav.* to 'nav'@'cluster0' identified by 'nav@123' with grant option;

# Cloudera Navigator Metastore Server:
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
grant all privileges on navms.* to 'navms'@'cluster0' identified by 'navms@123' with grant option;

# Oozie:
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
grant all privileges on oozie.* to 'oozie'@'cluster0' identified by 'oozie@123' with grant option;

flush privileges;

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'admin@2022' WITH GRANT OPTION;
FLUSH PRIVILEGES;

8. 主节点修改CM配置

sed -i "s/server_host=localhost/server_host=cluster0/g" /etc/cloudera-scm-agent/config.ini

vi /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=cluster0:3306
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=scm@123
com.cloudera.cmf.db.setupType=EXTERNAL

9. 其他节点部署cm agent不下载依赖包

rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
sed -i "s/server_host=localhost/server_host=cluster0/g" /etc/cloudera-scm-agent/config.ini

10. 开启cm server 与所有节点agent，并设置开机自启动

systemctl start cloudera-scm-server
systemctl enable cloudera-scm-server
systemctl status cloudera-scm-server

systemctl start cloudera-scm-agent
systemctl status cloudera-scm-agent
systemctl enable cloudera-scm-agent

Hadoop组件部署

• 访问地址：http://192.168.4.100:7180
• 账号密码：admin/admin
• CDHweb界面操作
• PHOENIXweb界面操作