k8s集群搭建过程详解
阅读原文时间:2022年03月30日阅读:2

准备工作

systemctl disable firewalld

systemctl stop firewalld

vim /etc/hostname

  • k8s-master01(对应主机ip:192.168.91.129)
  • k8s-worker01(对应主机ip:192.168.91.130)
  • k8s-worker02(对应主机ip:192.168.91.131)

配置3台测试机的/etc/hosts文件(在三台服务器上同步)

vim /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.91.129 k8s-master01

192.168.91.130 k8s-worker01

192.168.91.131 k8s-worker02

(主从节点都需要)wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

(主从节点都需要)wget https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

(主从节点都需要)rpm --import rpm-package-key.gpg

(主节点)yum repolist

(主节点)新建和编辑repo文件:

vim /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

(主节点)

cd /etc/yum.repos.d

scp CentOS-Base.repo docker-ce.repo kubernetes.repo k8s-worker01:/etc/yum.repos.d/

scp CentOS-Base.repo docker-ce.repo kubernetes.repo k8s-worker02:/etc/yum.repos.d/

安装Kubernetes

修改/etc/docker/daemon.json

添加:"exec-opts": ["native.cgroupdriver=systemd"](用于消除安装k8s集群时的警告)

systemctl daemon-reload

systemctl restart docker

更新缓存(主从节点都需要)

yum clean all

yum -y makecache

(若是执行第二步的时候报错:Cannot find a valid baseurl for repo: base/7/x86_64,则检查一下网络问题)

验证源是否可用

  yum list | grep kubeadm(如果提示要验证yum-key.gpg是否可用,输入y)

主从节点安装k8s-1.17.6:yum install -y kubelet-1.17.6-0 kubeadm-1.17.6-0 kubectl-1.17.6-0

查看相关配置:rpm -ql kubelet

其中,/etc/kubernetes/manifests 为清单目录,/etc/sysconfig/kubelet 为配置文件,/usr/lib/systemd/system/kubelet.service 为主程序。

增加配置信息:

vim /etc/sysconfig/kubelet

  KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"

  KUBELET_EXTRA_ARGS="--fail-swap-on=false"(直接关闭交换分区会导致虚拟机内存资源紧张,故直接配置忽略swap报错,这样在 kubeadm init 的时候,kubelet才能正常运行)

启动kubelet并设置开机启动(这里可以不执行启动命令,在主节点初始化时/从节点加入集群时会启动kubelet)

  systemctl enable kubelet

导入k8s及相关docker镜像

先手动将集群初始化镜像文件k8sv1.17.6.tar上传到各个节点的/root目录下

进入各个节点tar所在目录,使用如下命令导入镜像

docker load -i k8sv1.17.6.tar

先手动将集群网络组件calico(calico3.13.2.tar,一套开源的网络和网络安全方案)上传到各个节点的/root目录下,集群所有节点都需要导入

k8s集群需要的网络插件为3.13版本,集群所有节点都需要导入

cd /root

wget https://docs.projectcalico.org/v3.13/manifests/calico.yaml

若出现以下报错,则执行:yum install -y ca-certificates

初始化k8s集群

kubeadm init --apiserver-advertise-address=192.168.91.129 --kubernetes-version v1.17.6 --service-cidr=10.1.0.0/16 --pod-network-cidr=10.81.0.0/16 --image-repository=registry.aliyuncs.com/google_containers --ignore-preflight-errors=Swap

若出现以下错误:

W0222 21:17:31.258353 10153 validation.go:28] Cannot validate kube-proxy config - no validator is available

W0222 21:17:31.258577 10153 validation.go:28] Cannot validate kubelet config - no validator is available

[init] Using Kubernetes version: v1.17.6

[preflight] Running pre-flight checks

[WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly

[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

[WARNING Swap]: running with swap on is not supported. Please disable swap

[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.12. Latest validated version: 19.03

error execution phase preflight: [preflight] Some fatal errors occurred:

[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1

[ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1

[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=…

To see the stack trace of this error execute with --v=5 or higher

根据报错信息可知,是因为:/proc/sys/net/bridge/bridge-nf-call-iptables 和 /proc/sys/net/ipv4/ip_forward 的内容没有被设置为1

执行以下两条命令即可:

echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables

echo "1" >/proc/sys/net/ipv4/ip_forward

按照提示在master执行如下操作

  mkdir -p $HOME/.kube

  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

  sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubeadm join 192.168.91.129:6443 --token chijon.vjnq37z0titsxbmc --discovery-token-ca-cert-hash sha256:eca67b2c5ae245a6f203bc7de1030828e0f0d9fbdd5e9a640b83ab22a9d39609 --ignore-preflight-errors=Swap

若不加这段:--ignore-preflight-errors=Swap,则可能会出现如下报错:

[ERROR Swap]: running with swap on is not supported. Please disable swap

kubectl get nodes

由于还没有进行网络配置,故集群所有节点的状态为:NotReady

worker节点上面更是会直接报错:

运行 出现 The connection to the server localhost:8080 was refused - did you specify the right host or port? 的错误

  • 出现这个问题的原因是kubectl命令需要使用kubernetes-admin来运行,解决方法如下,将主节点中的【/etc/kubernetes/admin.conf】文件拷贝到从节点相同目录下,然后配置环境变量

  • echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile

  • source ~/.bash_profile

  • 如果某个节点的/etc/kubernetes目录下没有admin.conf这个配置文件,则直接将主节点上的配置文件复制过去

cd /root

kubectl apply -f calico.yaml

echo "source <(kubectl completion bash)" >> ~/.bash_profilesource ~/.bash_profile

kubectl get nodes

k8s-master01

可以看到集群所有节点的状态为:Ready

若某台虚拟机关机或断网了,则状态会变为 NotReady,重启或联网之后会自动加入集群,变为 Ready