http://blog.csdn.net/xu470438000/article/details/50512442
============================================================
2016-01-13 18:16 2005人阅读 评论(1) 收藏 举报
分类:
docker
版权声明:本文为博主原创文章,未经博主允许不得转载。
使用docker搭建部署hadoop分布式集群
在网上找了很长时间都没有找到使用docker搭建hadoop分布式集群的文档,没办法,只能自己写一个了。
一:环境准备:
1:首先要有一个Centos7操作系统,可以在虚拟机中安装。
2:在centos7中安装docker,docker的版本为1.8.2
安装步骤如下:
<1>安装制定版本的dockeryum install -y docker-1.8.2-10.el7.centos
<2>安装的时候可能会报错,需要删除这个依赖
rpm <span class="hljs-operator">-e</span> lvm2-<span class="hljs-number">7</span>:<span class="hljs-number">2.02</span>.<span class="hljs-number">105</span>-<span class="hljs-number">14</span>.el7.x86\_64
启动docker
service docker <span class="hljs-operator"><span class="hljs-keyword">start</span></span>
验证安装结果:
<3>启动之后执行docker info会看到下面有两行警告信息
需要关闭防火墙并重启系统
systemctl <span class="hljs-keyword">stop</span> firewalld
systemctl disable firewalld
注意:执行完上面的命令之后需要重启系统
reboot -h(重启系统)
<4>运行容器可能会报错
需要关闭selinux
解决方法:
<span class="hljs-attribute">1:setenforce 0(立刻生效,不需要重启操作系统)
2:修改/etc/selinux/config文件中的SELINUX</span>=<span class="hljs-string">disabled ,然后重启系统生效
建议两个步骤都执行,这样可以确保系统重启之后selinux也是关闭状态</span>
3:需要先构建一个hadoop的基础镜像,使用dockerfile文件方式进行构建。
先构建一个具备ssh功能的镜像,方便后期使用。(但是这样对于容器的安全性会有影响)
注意:这个镜像中的root用户的密码是root
Mkdir centos-ssh-root
Cd centos-ssh-root
Vi Dockerfile
<span class="hljs-preprocessor"># 选择一个已有的os镜像作为基础 </span>
FROM centos
<span class="hljs-preprocessor"># 镜像的作者 </span>
MAINTAINER crxy
<span class="hljs-preprocessor"># 安装openssh-server和sudo软件包,并且将sshd的UsePAM参数设置成no </span>
RUN yum install -y openssh-server sudo
RUN sed -i <span class="hljs-string">'s/UsePAM yes/UsePAM no/g'</span> /etc/ssh/sshd\_config
<span class="hljs-preprocessor">#安装openssh-clients</span>
RUN yum install -y openssh-clients
<span class="hljs-preprocessor"># 添加测试用户root,密码root,并且将此用户添加到sudoers里 </span>
RUN echo <span class="hljs-string">"root:root"</span> | chpasswd
RUN echo <span class="hljs-string">"root ALL=(ALL) ALL"</span> >> /etc/sudoers
<span class="hljs-preprocessor"># 下面这两句比较特殊,在centos6上必须要有,否则创建出来的容器sshd不能登录 </span>
RUN ssh-keygen -t dsa -f /etc/ssh/ssh\_host\_dsa\_key
RUN ssh-keygen -t rsa -f /etc/ssh/ssh\_host\_rsa\_key
<span class="hljs-preprocessor"># 启动sshd服务并且暴露22端口 </span>
RUN mkdir /<span class="hljs-keyword">var</span>/run/sshd
EXPOSE <span class="hljs-number">22</span>
CMD \[<span class="hljs-string">"/usr/sbin/sshd"</span>, <span class="hljs-string">"-D"</span>\]
构建命令:
docker build -t=”crxy/centos-ssh-root” .
查询刚才构建成功的镜像
4:基于这个镜像再构建一个带有jdk的镜像
注意:jdk使用的是1.7版本的
Mkdir centos-ssh-root-jdk
Cd centos-ssh-root-jdk
Cp ../jdk-7u75-linux-x64.tar.gz .
Vi Dockerfile
FROM crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span>
ADD jdk<span class="hljs-subst">-</span><span class="hljs-number">7</span>u75<span class="hljs-attribute">-linux</span><span class="hljs-attribute">-x64</span><span class="hljs-built\_in">.</span>tar<span class="hljs-built\_in">.</span>gz /usr/<span class="hljs-built\_in">local</span><span class="hljs-subst">/</span>
RUN mv /usr/<span class="hljs-built\_in">local</span>/jdk1<span class="hljs-number">.7</span><span class="hljs-number">.0</span>\_75 /usr/<span class="hljs-built\_in">local</span>/jdk1<span class="hljs-number">.7</span>
ENV JAVA\_HOME /usr/<span class="hljs-built\_in">local</span>/jdk1<span class="hljs-number">.7</span>
ENV PATH <span class="hljs-variable">$JAVA\_HOME</span>/bin:<span class="hljs-variable">$PATH</span>
构建命令:
docker build -t=”crxy/centos-ssh-root-jdk” .
查询构建成功的镜像
5:基于这个jdk镜像再构建一个带有hadoop的镜像
注意:hadoop使用的是2.4.1版本的。
Mkdir centos-ssh-root-jdk-hadoop
Cd centos-ssh-root-jdk-hadoop
Cp ../hadoop-2.4.1.tar.gz .
Vi Dockerfile
FROM crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span><span class="hljs-attribute">-jdk</span>
ADD hadoop<span class="hljs-subst">-</span><span class="hljs-number">2.4</span><span class="hljs-number">.1</span><span class="hljs-built\_in">.</span>tar<span class="hljs-built\_in">.</span>gz /usr/<span class="hljs-built\_in">local</span>
RUN mv /usr/<span class="hljs-built\_in">local</span>/hadoop<span class="hljs-subst">-</span><span class="hljs-number">2.4</span><span class="hljs-number">.1</span> /usr/<span class="hljs-built\_in">local</span>/hadoop
ENV HADOOP\_HOME /usr/<span class="hljs-built\_in">local</span>/hadoop
ENV PATH <span class="hljs-variable">$HADOOP\_HOME</span>/bin:<span class="hljs-variable">$PATH</span>
构建命令:
docker build -t=”crxy/centos-ssh-root-jdk-hadoop” .
查询构建成功的镜像
二:搭建hadoop分布式集群
1:集群规划
准备搭建一个具有三个节点的集群,一主两从
主节点:hadoop0 ip:192.168.2.10
从节点1:hadoop1 ip:192.168.2.11
从节点2:hadoop2 ip:192.168.2.12
但是由于docker容器重新启动之后ip会发生变化,所以需要我们给docker设置固定ip。使用pipework给docker容器设置固定ip
2:启动三个容器,分别作为hadoop0 hadoop1 hadoop2
在宿主机上执行下面命令,给容器设置主机名和容器的名称,并且在hadoop0中对外开放端口50070 和8088
docker run <span class="hljs-subst">--</span>name hadoop0 <span class="hljs-subst">--</span>hostname hadoop0 <span class="hljs-attribute">-d</span> <span class="hljs-attribute">-P</span> <span class="hljs-attribute">-p</span> <span class="hljs-number">50070</span>:<span class="hljs-number">50070</span> <span class="hljs-attribute">-p</span> <span class="hljs-number">8088</span>:<span class="hljs-number">8088</span> crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span><span class="hljs-attribute">-jdk</span><span class="hljs-attribute">-hadoop</span>
docker run <span class="hljs-subst">--</span>name hadoop1 <span class="hljs-subst">--</span>hostname hadoop1 <span class="hljs-attribute">-d</span> <span class="hljs-attribute">-P</span> crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span><span class="hljs-attribute">-jdk</span><span class="hljs-attribute">-hadoop</span>
docker run <span class="hljs-subst">--</span>name hadoop2 <span class="hljs-subst">--</span>hostname hadoop2 <span class="hljs-attribute">-d</span> <span class="hljs-attribute">-P</span> crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span><span class="hljs-attribute">-jdk</span><span class="hljs-attribute">-hadoop</span>
使用docker ps 查看刚才启动的是三个容器
3:给这三台容器设置固定IP
1:下载pipework
下载地址:https://github.com/jpetazzo/pipework.git
2:把下载的zip包上传到宿主机服务器上,解压,改名字
unzip pipework<span class="hljs-attribute">-master</span><span class="hljs-built\_in">.</span>zip
mv pipework<span class="hljs-attribute">-master</span> pipework
cp <span class="hljs-attribute">-rp</span> pipework/pipework /usr/<span class="hljs-built\_in">local</span>/bin<span class="hljs-subst">/</span>
3:安装bridge-utils
yum <span class="hljs-attribute">-y</span> install bridge<span class="hljs-attribute">-utils</span>
4:创建网络
brctl addbr br0
ip link <span class="hljs-operator"><span class="hljs-keyword">set</span> dev br0 up
ip addr <span class="hljs-keyword">add</span> <span class="hljs-number">192.168</span><span class="hljs-number">.2</span><span class="hljs-number">.1</span>/<span class="hljs-number">24</span> dev br0</span>
5:给容器设置固定ip
<span class="hljs-title">pipework</span> br0 hadoop0 <span class="hljs-number">192.168.2.10</span>/<span class="hljs-number">24</span>
pipework br0 hadoop1 <span class="hljs-number">192.168.2.11</span>/<span class="hljs-number">24</span>
pipework br0 hadoop2 <span class="hljs-number">192.168.2.12</span>/<span class="hljs-number">24</span>
验证一下,分别ping三个ip,能ping通就说明没问题
4:配置hadoop集群
先连接到hadoop0上,
使用命令
docker <span class="hljs-keyword">exec</span> -it hadoop0 /bin/bash
下面的步骤就是hadoop集群的配置过程
1:设置主机名与ip的映射,修改三台容器:vi /etc/hosts
添加下面配置
192.168.2.10 hadoop0
192.168.2.11 hadoop1
192.168.2.12 hadoop2
2:设置ssh免密码登录
在hadoop0上执行下面操作
cd ~
mkdir <span class="hljs-built\_in">.</span>ssh
cd <span class="hljs-built\_in">.</span>ssh
ssh<span class="hljs-attribute">-keygen</span> <span class="hljs-attribute">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> localhost
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> hadoop0
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> hadoop1
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> hadoop2
在hadoop1上执行下面操作
cd ~
cd <span class="hljs-built\_in">.</span>ssh
ssh<span class="hljs-attribute">-keygen</span> <span class="hljs-attribute">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> localhost
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> hadoop1
在hadoop2上执行下面操作
cd ~
cd <span class="hljs-built\_in">.</span>ssh
ssh<span class="hljs-attribute">-keygen</span> <span class="hljs-attribute">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> localhost
ssh<span class="hljs-attribute">-copy</span><span class="hljs-attribute">-id</span> <span class="hljs-attribute">-i</span> hadoop2
3:在hadoop0上修改hadoop的配置文件
进入到/usr/local/hadoop/etc/hadoop目录
修改目录下的配置文件core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml
(1)hadoop-env.sh
<span class="hljs-keyword">export</span> JAVA\_HOME=/usr/local/jdk1.<span class="hljs-number">7</span>
(2)core-site.xml
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>fs.defaultFS<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>hdfs://hadoop0:9000<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>hadoop.tmp.dir<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>/usr/local/hadoop/tmp<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>fs.trash.interval<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>1440<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
(3)hdfs-site.xml
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>dfs.replication<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>1<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>dfs.permissions<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>false<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
(4)yarn-site.xml
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>mapreduce\_shuffle<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.log-aggregation-enable<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>true<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
(5)修改文件名:mv mapred-site.xml.template mapred-site.xml
vi mapred-site.xml
<span class="hljs-tag"><<span class="hljs-title">configuration</span>></span>
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>mapreduce.framework.name<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>yarn<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
<span class="hljs-tag"></<span class="hljs-title">configuration</span>></span>
(6)格式化
进入到/usr/local/hadoop目录下
1、执行格式化命令
bin/hdfs namenode <span class="hljs-attribute">-format</span>
注意:在执行的时候会报错,是因为缺少which命令,安装即可
执行下面命令安装
yum install <span class="hljs-attribute">-y</span> which
看到下面命令说明格式化成功。
格式化操作不能重复执行。如果一定要重复格式化,带参数-force即可。
(7)启动伪分布hadoop
命令:sbin/<span class="hljs-operator"><span class="hljs-keyword">start</span>-<span class="hljs-keyword">all</span>.sh</span>
第一次启动的过程中需要输入yes确认一下。
使用jps,检查进程是否正常启动?能看到下面几个进程表示伪分布启动成功
\[root<span class="hljs-variable">@hadoop0</span> hadoop\]<span class="hljs-comment"># jps</span>
<span class="hljs-number">3267</span> <span class="hljs-constant">SecondaryNameNode</span>
<span class="hljs-number">3003</span> <span class="hljs-constant">NameNode</span>
<span class="hljs-number">3664</span> <span class="hljs-constant">Jps</span>
<span class="hljs-number">3397</span> <span class="hljs-constant">ResourceManager</span>
<span class="hljs-number">3090</span> <span class="hljs-constant">DataNode</span>
<span class="hljs-number">3487</span> <span class="hljs-constant">NodeManager</span>
(8)停止伪分布hadoop
命令:sbin/stop-<span class="hljs-built\_in">all</span>.sh
(9)指定nodemanager的地址,修改文件yarn-site.xml
<span class="hljs-tag"><<span class="hljs-title">property</span>></span>
<span class="hljs-tag"><<span class="hljs-title">description</span>></span>The hostname of the RM.<span class="hljs-tag"></<span class="hljs-title">description</span>></span>
<span class="hljs-tag"><<span class="hljs-title">name</span>></span>yarn.resourcemanager.hostname<span class="hljs-tag"></<span class="hljs-title">name</span>></span>
<span class="hljs-tag"><<span class="hljs-title">value</span>></span>hadoop0<span class="hljs-tag"></<span class="hljs-title">value</span>></span>
<span class="hljs-tag"></<span class="hljs-title">property</span>></span>
(10)修改hadoop0中hadoop的一个配置文件etc/hadoop/slaves
删除原来的所有内容,修改为如下
hadoop1
hadoop2
(11)在hadoop0中执行命令
scp <span class="hljs-attribute">-rq</span> /usr/<span class="hljs-built\_in">local</span>/hadoop hadoop1:/usr/<span class="hljs-built\_in">local</span>
scp <span class="hljs-attribute">-rq</span> /usr/<span class="hljs-built\_in">local</span>/hadoop hadoop2:/usr/<span class="hljs-built\_in">local</span>
(12)启动hadoop分布式集群服务
执行sbin/<span class="hljs-operator"><span class="hljs-keyword">start</span>-<span class="hljs-keyword">all</span>.sh</span>
注意:在执行的时候会报错,是因为两个从节点缺少which命令,安装即可
分别在两个从节点执行下面命令安装
yum <span class="hljs-keyword">install</span> -y which
再启动集群(如果集群已启动,需要先停止)
sbin/<span class="hljs-operator"><span class="hljs-keyword">start</span>-<span class="hljs-keyword">all</span>.sh</span>
(13)验证集群是否正常
首先查看进程:
Hadoop0上需要有这几个进程
\[root<span class="hljs-variable">@hadoop0</span> hadoop\]<span class="hljs-comment"># jps</span>
<span class="hljs-number">4643</span> <span class="hljs-constant">Jps</span>
<span class="hljs-number">4073</span> <span class="hljs-constant">NameNode</span>
<span class="hljs-number">4216</span> <span class="hljs-constant">SecondaryNameNode</span>
<span class="hljs-number">4381</span> <span class="hljs-constant">ResourceManager</span>
Hadoop1上需要有这几个进程
\[root<span class="hljs-variable">@hadoop1</span> hadoop\]<span class="hljs-comment"># jps</span>
<span class="hljs-number">715</span> <span class="hljs-constant">NodeManager</span>
<span class="hljs-number">849</span> <span class="hljs-constant">Jps</span>
<span class="hljs-number">645</span> <span class="hljs-constant">DataNode</span>
Hadoop2上需要有这几个进程
\[root<span class="hljs-variable">@hadoop2</span> hadoop\]<span class="hljs-comment"># jps</span>
<span class="hljs-number">456</span> <span class="hljs-constant">NodeManager</span>
<span class="hljs-number">589</span> <span class="hljs-constant">Jps</span>
<span class="hljs-number">388</span> <span class="hljs-constant">DataNode</span>
使用程序验证集群服务
创建一个本地文件
vi <span class="hljs-operator">a</span>.txt
hello you
hello me
上传a.txt到hdfs上
hdfs dfs -<span class="hljs-built\_in">put</span> <span class="hljs-operator">a</span>.txt /
执行wordcount程序
cd /usr/local/hadoop/share/hadoop/mapreduce
hadoop jar hadoop-mapreduce-examples-<span class="hljs-number">2.4</span><span class="hljs-number">.1</span><span class="hljs-preprocessor">.jar</span> wordcount /a<span class="hljs-preprocessor">.txt</span> /<span class="hljs-keyword">out</span>
查看程序执行结果
这样就说明集群正常了。
通过浏览器访问集群的服务
由于在启动hadoop0这个容器的时候把50070和8088映射到宿主机的对应端口上了
adb9eba7142b crxy/centos<span class="hljs-attribute">-ssh</span><span class="hljs-attribute">-root</span><span class="hljs-attribute">-jdk</span><span class="hljs-attribute">-hadoop</span> <span class="hljs-string">"/usr/sbin/sshd -D"</span> About an hour ago Up About an hour <span class="hljs-number">0.0</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>:<span class="hljs-number">8088</span><span class="hljs-subst">-></span><span class="hljs-number">8088</span>/tcp, <span class="hljs-number">0.0</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>:<span class="hljs-number">50070</span><span class="hljs-subst">-></span><span class="hljs-number">50070</span>/tcp, <span class="hljs-number">0.0</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>:<span class="hljs-number">32770</span><span class="hljs-subst">-></span><span class="hljs-number">22</span>/tcp hadoop0
所以在这可以直接通过宿主机访问容器中hadoop集群的服务
宿主机的ip为:192.168.1.144
http:<span class="hljs-comment">//192.168.1.144:50070/</span>
http:<span class="hljs-comment">//192.168.1.144:8088/</span>
三:集群节点重启
停止三个容器,在宿主机上执行下面命令
docker <span class="hljs-keyword">stop</span> hadoop0
docker <span class="hljs-keyword">stop</span> hadoop1
docker <span class="hljs-keyword">stop</span> hadoop2
容器停止之后,之前设置的固定ip也会消失,重新再使用这几个容器的时候还需要重新设置固定ip
先把之前停止的三个容器起来
docker <span class="hljs-operator"><span class="hljs-keyword">start</span> hadoop0
docker <span class="hljs-keyword">start</span> hadoop1
docker <span class="hljs-keyword">start</span> hadoop2</span>
在宿主机上执行下面命令重新给容器设置固定ip
<span class="hljs-title">pipework</span> br0 hadoop0 <span class="hljs-number">192.168.2.10</span>/<span class="hljs-number">24</span>
pipework br0 hadoop1 <span class="hljs-number">192.168.2.11</span>/<span class="hljs-number">24</span>
pipework br0 hadoop2 <span class="hljs-number">192.168.2.12</span>/<span class="hljs-number">24</span>
还需要重新在容器中配置主机名和ip的映射关系,每次都手工写比较麻烦
写一个脚本,runhosts.sh
<span class="hljs-shebang">#!/bin/bash</span>
<span class="hljs-built\_in">echo</span> <span class="hljs-number">192.168</span>.<span class="hljs-number">2.10</span> hadoop0 >> /etc/hosts
<span class="hljs-built\_in">echo</span> <span class="hljs-number">192.168</span>.<span class="hljs-number">2.11</span> hadoop1 >> /etc/hosts
<span class="hljs-built\_in">echo</span> <span class="hljs-number">192.168</span>.<span class="hljs-number">2.12</span> hadoop2 >> /etc/hosts
添加执行权限,chmod +x runhosts.sh
把这个脚本拷贝到所有节点,并且分别执行这个脚本
scp <span class="hljs-filename">runhosts.sh hadoop1</span>:~
scp <span class="hljs-filename">runhosts.sh hadoop2</span>:~
执行脚本的命令 ./runhosts.sh
查看/etc/hosts文件中是否添加成功
注意:有一些docker版本中不会在hosts文件中自动生成下面这些映射,所以我们才在这里手工给容器设置固定ip,并设置主机名和ip的映射关系。
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.25</span> hadoop0
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.25</span> hadoop0<span class="hljs-preprocessor">.bridge</span>
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.26</span> hadoop1
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.26</span> hadoop1<span class="hljs-preprocessor">.bridge</span>
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.27</span> hadoop2
<span class="hljs-number">172.17</span><span class="hljs-number">.0</span><span class="hljs-number">.27</span> hadoop2<span class="hljs-preprocessor">.bridge</span>
启动hadoop集群
sbin/<span class="hljs-operator"><span class="hljs-keyword">start</span>-<span class="hljs-keyword">all</span>.sh</span>
手机扫一扫
移动阅读更方便
你可能感兴趣的文章