k8s-2-集成apollo配置中心
主题: 在k8s中集成Apollo配置中心
一、配置中心概述
配置的几种方式
本课讲得是基于配置中心数据库实现
配置管理的现状
常见的配置中心
主讲:k8s configmap,apollo
二、实战k8s配置中心-configmap
2.1.1: Configmap说明
**configmap和secret
**
在K8S中,有两种方式管理资源的配置,分别是configmap和secret,他们的最大区别是:
configmap用来管理明文配置
secret用来管理密文配置
**0.1.2 怎么使用configmap
**
创建configmap资源
在deploy中定义congfigmap类型的volumes
在deoloy的containers中挂载此volumes
0.1.3 更新configmap配置的方法
单配置更新法:
修改configmap资源清单并应用
重启使用该configmap资源的pod
多配置更换法:
准备多个configmap资源
修改dp中使用的configmap
重新apply该dp,pod会自动重启
2.1.2: Configmap使用准备
清理资源
先将前面部署的3个dubbo服务的POD个数全部调整(scale)为0个,避免在应用configmap过程中可能的报错,也为了节省资源
直接在dashboard上操作即可,
App命名空间--à生产者和消费者都伸缩成0
Dubbo-monitor也设置为0
拆分zk集群
**分测试和生产环境了,将原来3台zk集群连接断开,将zk1,和zk2做成单独的zk使用,zk3关掉
**
主机名
角色
ip
HDSS7-11.host.com
zk1.od.com(Test环境)
10.4.7.11
HDSS7-12.host.com
zk2.od.com(Prod环境)
10.4.7.12
**先关闭zk集群
**
/opt/zookeeper/bin/zkServer.sh stop (3台zk集群全部关闭)
**#删除11,12节点上zookeeper上的数据和日志
**
[root@hdss7-12 ~]# rm /data/zookeeper/data/* -rf
[root@hdss7-12 ~]# rm /data/zookeeper/logs/* -rf
**#修改配置文件.删除与其他主机通信(11,12主机上)
**
保留如下这些
[root@hdss7-12 ~]# cat /opt/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
clientPort=2181
**重新开启(11,12)
**
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh start
dubbo-monitor资源配置清单
老规矩,资源清单在7.200运维机上统一操作
cd /data/k8s-yaml/dubbo-monitor
configmap
创建comfigmap清单
cat >cm.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
name: dubbo-monitor-cm
namespace: infra
data:
dubbo.properties: |
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=OldboyEdu
dubbo.registry.address=zookeeper://zk1.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=/dubbo-monitor-simple/logs/dubbo-monitor.log
dubbo.log4j.level=WARN
EOF
其实就是把dubbo-monitor配置文件中的内容用configmap语法展示出来了
当然最前面加上了相应的元数据信息
Configmap作为一个卷挂载到了容器里
deploy
cat >dp-cm.yaml <<'EOF'
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-monitor
namespace: infra
labels:
name: dubbo-monitor
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-monitor
template:
metadata:
labels:
app: dubbo-monitor
name: dubbo-monitor
spec:
containers:
- name: dubbo-monitor
image: harbor.od.com/infra/dubbo-monitor:latest
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
imagePullPolicy: IfNotPresent
#----------------start---------------------------
volumeMounts:
- name: configmap-volume
mountPath: /dubbo-monitor-simple/conf
volumes:
- name: configmap-volume
configMap: #挂载到了configmap-volume目录上,容器中目录
name: dubbo-monitor-cm
#----------------end-----------------------------
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
EOF
注释中的内容就是新增在原dp.yaml中增加的内容,解释如下:
申明一个卷,卷名为configmap-volume
指定这个卷使用名为dubbo-monitor-cm的configMap
在containers中挂载卷,卷名与申明的卷相同
用mountPath的方式挂载到指定目录
**Umask使挂载只读
**
创建资源并检查
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/cm.yaml
kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/dp-cm.yaml
dashboard检查创建结果
在dashboard中查看infra名称空间中的configmap资源
然后检查容器中的配置
kubectl -n infra exec -it dubbo-monitor-5b7cdddbc5-xpft6 bash
# 容器内
bash-4.3# cat /dubbo-monitor-simple/conf/dubbo.properties
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=
dubbo.registry.address=zookeeper://zk1.od.com:2181
….
http://dubbo-monitor.od.com/status.html
检查dubbo-monitor页面的注册信息
2.1.3: 更新configmap资源
多配置更新法
准备新configmap
再准备一个configmap叫cm-pro.yaml
cp cm.yaml cm-pro.yaml
# 把资源名字改成dubbo-monitor-cm-pro
sed -i 's#dubbo-monitor-cm#dubbo-monitor-cm-pro#g' cm-pro.yaml
**# 把服务注册到zk2.od.com上
**
sed -i 's#zk1#zk2#g' cm-pro.yaml
dashboard上修改zook地址
修改deploy配置
sed -i 's#dubbo-monitor-cm#dubbo-monitor-cm-pro#g' dp-cm.yaml
更新资源
# 应用新configmap
kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/cm-pro.yaml
# 更新deploy
kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/dp-cm.yaml
检查配置是否更新
新的pod已经起来了
~]# kubectl -n infra get pod
NAME READY STATUS RESTARTS AGE
dubbo-monitor-c7fbf68b9-7nffj 1/1 Running 0 52s
进去看看是不是应用的新的configmap配置：
kubectl -n infra exec -it dubbo-monitor-5cb756cc6c-xtnrt bash
# 容器内
bash-4.3# cat /dubbo-monitor-simple/conf/dubbo.properties |grep zook
dubbo.registry.address=zookeeper://zk2.od.com:2181
看下dubbo-monitor的页面：已经是zk2了。
挂载方式探讨
monutPath挂载的问题
我们使用的是mountPath，这个是挂载整个目录，会使容器内的被挂载目录中原有的文件不可见，可以看见我们。
查看我们pod容器启动的命令可以看见原来脚本中的命令已经无法对挂载的目录操作了
如何单独挂载一个配置文件:
只挂载单独一个文件而不是整个目录，需要添加subPath方法
单独挂载文件演示
更新配置
在dp-cm.yaml的配置中,将原来的volume配置做一下更改
#----------------start---------------------------
volumeMounts:#卷挂载动作
- name: configmap-volume
mountPath: /dubbo-monitor-simple/conf
volumes:#声明卷
- name: configmap-volume
configMap:
name: dubbo-monitor-cm
#----------------end-----------------------------
# 调整为
#----------------start---------------------------
volumeMounts:
- name: configmap-volume
mountPath: /dubbo-monitor-simple/conf
- name: configmap-volume
mountPath: /var/dubbo.properties
subPath: dubbo.properties
volumes:
- name: configmap-volume
configMap:
name: dubbo-monitor-cm
#----------------end-----------------------------
应用apply配置并验证
kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/dp-cm.yaml
kubectl -n infra exec -it dubbo-monitor-5cb756cc6c-xtnrt bash
# 容器内操作
bash-4.3# ls -l /var/
total 4
drwxr-xr-x 1 root root 29 Apr 13 2016 cache
-rw-r--r-- 1 root root 459 May
10 10:02 dubbo.properties
drwxr-xr-x 2 root root 6 Apr 1 2016 empty
…..
根据文件创建configmap资源
[root@hdss7-22 conf]# kubectl create cm kubelet-cm --from-file=./kubelet.kubeconfig
三、Apollo配置中心介绍
3.1: 介绍
3.2: 架构
Client: dubbo-demo-custom,dubbo-demo-service
Apoll3兄弟:config service,admin service,portal
将dubbo的配置,写到config DB
Portal:web界面,可以在页面修改configdb中的配置数据
四、实战交付apollo配置中心组件---configservice到k8s集群
Apollo的注册中心
4.1: apollo-configservice安装部署
4.1.1: 准备apollo-configservice软件包
apollo-configservice软件包下载地址：https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip
在运维主机200上执行
[root@mfyxw50 ~]# cd /opt/src
[root@mfyxw50 src]# wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip
[root@mfyxw50 src]# mkdir -p /data/dockerfile/apollo-configservice
[root@mfyxw50 src]# unzip apollo-configservice-1.5.1-github.zip -d /data/dockerfile/apollo-configservice/
[root@mfyxw50 src]# rm -fr /data/dockerfile/apollo-configservice/apollo-configservice-1.5.1-sources.jar #用不到
[root@hdss7-200 scripts]# rm /data/dockerfile/apollo-configservice/scripts/shutdown.sh -f #用不到
#apollo-configservice-1.5.1-sources.jar源码包用不到
4.1.2: 安装MariaDB数据库
在数据库主机HDSS7-11.host.com上：
注意：MySQL版本应为5.6或以上！
(1)添加MariaDB源
vi /etc/yum.repos.d/MariaDB.repo
[mariadb]
name = MariaDB
baseurl = https://mirrors.ustc.edu.cn/mariadb/yum/10.1/centos7-amd64/
gpgkey=https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
gpgcheck=1
(2)导入MariaDB证书
导入GPG-KEY
[root@hdss7-11 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
(3)生成缓存
[root@mfyxw10 ~]# yum makecache
(4)查看可用的MariaDB数据库版本
[root@mfyxw10 ~]# yum
list
MariaDB-server
--show-duplicates
Loaded
plugins: fastestmirror
Loading
mirror
speeds
from
cached
hostfile
* base: mirrors.163.com
* extras: mirrors.ustc.edu.cn
* updates: mirrors.aliyun.com
Available
Packages
MariaDB-server.x86_64 10.1.43-1.el7.centos
mariadb
MariaDB-server.x86_64 10.1.44-1.el7.centos
mariadb
MariaDB-server.x86_64 10.1.45-1.el7.centos
mariadb
mariadb-server.x86_64 1:5.5.65-1.el7
(5)安装MariaDB-Server 10.1.45版本
[root@mfyxw10 ~]# yum -y install MariaDB-server
(6)设置MariaDB配置文件
/etc/my.cnf.d/server.cnf文件内容如下
[root@mfyxw10 ~]# cat > /etc/my.cnf.d/server.cnf << EOF
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
character_set_server = utf8mb4
collation_server = utf8mb4_general_ci
init_connect = "SET NAMES 'utf8mb4'"
#
# * Galera-related settings
#
[galera]
# Mandatory settings
#wsrep_on=ON
#wsrep_provider=
#wsrep_cluster_address=
#binlog_format=row
#default_storage_engine=InnoDB
#innodb_autoinc_lock_mode=2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
EOF
/etc/my.cnf.d/mysql-clients.cnf文件内容如下
[root@mfyxw10 ~]# cat > /etc/my.cnf.d/mysql-clients.cnf << EOF
#
# These groups are read by MariaDB command-line tools
# Use it for options that affect only one utility
#
[mysql]
default-character-set = utf8mb4
[mysql_upgrade]
[mysqladmin]
[mysqlbinlog]
[mysqlcheck]
[mysqldump]
[mysqlimport]
[mysqlshow]
[mysqlslap]
EOF
(7)启动MariaDB数据库并添加至开机自启
[root@mfyxw10 ~]# systemctl enable --now mariadb
[root@hdss7-11 ~]# systemctl start mysql
[root@hdss7-11 ~]# systemctl status mysql
[root@mfyxw10 ~]# netstat -tanlp | grep mysql #查看MariaDB的启动端口
(8)设置MariaDB数据库密码
[root@mfyxw10 ~]# mysqladmin -uroot password #设置密码：回车
[root@mfyxw10 ~]# mysql -uroot -p #输入数据库密码登录
MariaDB [(none)]> \s #检查字符集
MariaDB [(none)]> drop database test;
MariaDB [(none)]> use mysql;
MariaDB [mysql]> delete from user where user='';
或都使用如下命令对MariaDB数据库初始化设置
[root@mfyxw10 ~]# mysql_secure_installation
(9)登录MariaDB数据库查看编码是否都是UTF-8
[root@mfyxw10 ~]# mysql
-uroot
-p
MariaDB
[(none)]> \s
(10)下载并导入apollo数据库初始化脚本
apollo的初始化数据脚本：https://raw.githubusercontent.com/ctripcorp/apollo/v1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql
[root@mfyxw10 ~]# wget https://raw.githubusercontent.com/ctripcorp/apollo/v1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql -O apolloconfig.sql
[root@mfyxw10 ~]# mysql -uroot -p < apolloconfig.sql
(11)查看apollo的数据库及表
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> show databases;
MariaDB [(none)]> use ApolloConfigDB;
MariaDB [ApolloConfigDB]> show tables;
(12)给数据库用户授予权限
[root@mfyxw10 ~]# mysql -uroot -p
Query OK, 0 rows affected (0.00 sec)
MariaDB [mysql]> select user,host from mysql.user;
+--------------+-------------------+
+--------------+-------------------+
| apolloconfig | 192.168.80.% |
+--------------+-------------------+
为什么写"@"10.4.7.%"授权网段, 因为数据库在11主机上,没有做flannel,出网进网肯定是要用10.4.7网段才能访问.
(13)修改ApolloConfigDB数据库的值
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> use ApolloConfigDB;
MariaDB [ApolloConfigDB]> select * from ServerConfig\G;
[root@mfyxw10 ~]# mysql -uroot -p
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
MariaDB [ApolloConfigDB]> select * from ServerConfig\G;
4.1.3: 解析域名
(1)在od.com域名的配置文件中添加mysql.od.com和config.od.com域名
[root@mfyxw10 ~]# cat > /var/named/od.com.zone << EOF
@ IN SOA dns.od.com. dnsadmin.od.com. (
harbor A 192.168.80.50 ;添加harbor记录
dubbo-monitor A 192.168.80.100
(2)重启DNS服务器服务
[root@mfyxw10 ~]# systemctl restart named
(3)测试域名解析
[root@hdss7-11 ~]# dig -t A config.od.com @10.4.7.11 +short 192.168.80.11
4.1.4: 制作apolloconfig的docker镜像
(1)更新application-github.properties
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
#apollo.eureka.server.enabled=true
#apollo.eureka.client.enabled=true
[**(2)更新startup.sh文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-configservice/scripts/startup.sh << EOF
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
## Adjust server port if necessary
LOG_DIR=~/opt/logs/apollo-config-server
APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)
SERVER_URL="http://${APOLLO_CONFIG_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
elif type -p java > /dev/null 2>&1; then
elif [[ -x "/usr/bin/java" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
printf "$(date) ==== Starting ==== \n"
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
[root@hdss7-200 scripts]# chmod u+x startup.sh
[root@hdss7-21 ~]# kubectl logs apollo-configservice-746fd46575-w8sns -n infra
(3)编写Dockefile文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-configservice/Dockerfile << EOF
FROM harbor.od.com/base/jre8:8u112
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-\${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"]
(4)制作docker镜像
[root@mfyxw50 ~]# cd /data/dockerfile/apollo-configservice
(5)将制作好的docker镜像上传至私有仓库
[root@mfyxw50 ~]# docker login harbor.od.com
[root@mfyxw50 ~]# docker push harbor.od.com/infra/apollo-configservice:v1.5.1
(6)查看私有仓库中infra是否已经有apollo-configservice:v1.5.1
4.1.5: 提供apolloconfig配置资源清单
(1)创建存储apolloconfig配置资源清单的目录
[root@mfyxw50 ~]# mkdir -p /data/k8s-yaml/apollo-configservice
(2)添加配置资源清单
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/deployment.yaml << EOF
apiVersion: extensions/v1beta1
image: harbor.od.com/infra/apollo-configservice:v1.5.1
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
terminationGracePeriodSeconds: 30
schedulerName: default-scheduler
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/service.yaml << EOF
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/Ingress.yaml << EOF
apiVersion: extensions/v1beta1
serviceName: apollo-configservice
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-configservice/configmap.yaml << EOF
application-github.properties: |
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.od.com/eureka
4.1.6: 应用apolloconfig配置资源清单
(1)应用apolloconfig配置资源清单
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/configmap.yaml
configmap/apollo-configservice-cm created
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/deployment.yaml
deployment.extensions/apollo-configservice created
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/service.yaml
service/apollo-configservice created
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/Ingress.yaml
ingress.extensions/apollo-configservice created
(2)查看apolloconfig的pod是否运行起来
[root@mfyxw30 ~]# kubectl get pod -n infra
NAME READY STATUS RESTARTS AGE
apollo-configservice-5f6555448-wssq5 1/1 Running 0 51s
dubbo-monitor-6676dd74cc-9hghb 1/1 Running 7 14d
dubbo-monitor-6676dd74cc-rd86g 1/1 Running 6 14d
jenkins-b99776c69-p6skp 1/1 Running 14 36d
4.1.7: 浏览器访问config.od.com
4.2: apollo-configservice连接数据库IP分析
五、交付apollo配置中心组件---adminservice到k8s集群
在Portal的web界面创建项目,写入键值,相当于在dashboard修改了configmap中的配置
Portal会把数据传递给adminservice,adminservice又把数据存到configdb中
在pod的dp中指定获取数据路径.指定config.od.com.当删除重建pod时会在这个地址读取定义的变量数据
apollo-adminservice软件包链接地址：https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-adminservice-1.5.1-github.zip
(1)下载软件
[root@mfyxw50 ~]# cd /opt/src/
(2)解压到指定的目录
[root@mfyxw50 ~]# mkdir -p /data/dockerfile/apollo-adminservice
(3)删除不使用的文件
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-adminservice/apollo-adminservice-1.5.1-sources.jar
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-adminservice/scripts/shutdown.sh
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-adminservice/apollo-adminservice.conf
(1)配置数据库连接串
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
(2)更新startup.sh文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-adminservice/scripts/startup.sh << EOF
SERVICE_NAME=apollo-adminservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-adminservice
## Adjust server port if necessary
APOLLO_ADMIN_SERVICE_NAME=\$(hostname -i)
# SERVER_URL="http://localhost:\${SERVER_PORT}"
SERVER_URL="http://\${APOLLO_ADMIN_SERVICE_NAME}:\${SERVER_PORT}"
## Adjust memory settings if necessary
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="\$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
if [[ -n "\$JAVA_HOME" ]] && [[ -x "\$JAVA_HOME/bin/java" ]]; then
javaexe="\$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
elif [[ -x "/usr/bin/java" ]]; then
version=\$("\$javaexe" -version 2>&1 | awk -F '"' '/version/ {print \$2}')
version=\$(echo "\$version" | awk -F. '{printf("%03d%03d",\$1,\$2);}')
# now version is of format 009003 (9.3.x)
if [ \$version -ge 011000 ]; then
elif [ \$version -ge 010000 ]; then
elif [ \$version -ge 009000 ]; then
JAVA_OPTS="\$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="\$JAVA_OPTS -Xloggc:\$LOG_DIR/gc.log -XX:+PrintGCDetails"
printf "\$(date) ==== Starting ==== \n"
chmod 755 \$SERVICE_NAME".jar"
echo "\$(date) Failed to start \$SERVICE_NAME.jar, return code: \$rc"
(3)编写apollo-adminservice的Dockerfile文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-adminservice/Dockerfile << EOF
FROM harbor.od.com/base/jre8:8u112
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-adminservice-\${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
ADD config/ /apollo-adminservice/config
ADD scripts/ /apollo-adminservice/scripts
CMD ["/apollo-adminservice/scripts/startup.sh"]
(3)制作镜像并推送
[root@mfyxw50 ~]# cd /data/dockerfile/apollo-adminservice
[root@mfyxw50 apollo-adminservice]# docker build . -t harbor.od.com/infra/apollo-adminservice:v1.5.1
[root@mfyxw50 ~]# docker login harbor.od.com
[root@mfyxw50 ~]# docker push harbor.od.com/infra/apollo-adminservice:v1.5.1
(4)登录harbor查看是否已经上传成功
[root@mfyxw50 ~]# mkdir -p /data/k8s-yaml/apollo-adminservice
(1)Deployment.yaml文件内容如下
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-adminservice/deployment.yaml << EOF
apiVersion: extensions/v1beta1
image: harbor.od.com/infra/apollo-adminservice:v1.5.1
mountPath: /apollo-adminservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
terminationGracePeriodSeconds: 30
schedulerName: default-scheduler
(2)ConfigMap.yaml文件内容如下
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-adminservice/configmap.yaml << EOF
application-github.properties: |
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.od.com/eureka
在master节点(mfyxw30.mfyxw.com或mfyxw40.mfyxw.com)任意一台操作
(1)应用资源配置清单
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/deployment.yaml
deployment.extensions/apollo-adminservice created
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/configmap.yaml
configmap/apollo-adminservice-cm created
(2)查看apollo-adminservice的pod是否运行成功
[root@mfyxw30 ~]# kubectl get pod -n infra
NAME READY STATUS RESTARTS AGE
apollo-adminservice-5cccf97c64-bhqzb 1/1 Running 0 72s
apollo-configservice-5f6555448-wwgsc 1/1 Running 1 2d17h
dubbo-monitor-6676dd74cc-9hghb 1/1 Running 11 17d
dubbo-monitor-6676dd74cc-rd86g 1/1 Running 10 17d
(3)浏览器访问config.od.com是否出现有apollo-adminservice
六、交付apollo配置中心组件---portal到k8s集群
(1)将apollo-portal数据库脚本下载回来
[root@mfyxw10 ~]# wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql -O apolloportal.sql
(2)导入apollo-portal数据库脚本
[root@mfyxw10 ~]# mysql -uroot -p < apolloportal.sql
(3)查看是否导入成功
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> show databases; #Apollo-portal的数据库名为：ApolloPortalDB
MariaDB [(none)]> use ApolloPortalDB; #切换至ApolloPortalDB数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
MariaDB [ApolloPortalDB]> show tables; #查看ApolloPortalDB数据库下的数据表
(4)ApolloPortal数据库授权
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]>flush privileges;
Query OK, 0 rows affected (0.00 sec)
(5)更新ApolloPortal数据表ServerConfig的Value
[root@mfyxw10 ~]# mysql -uroot -p
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 44
Server version: 10.1.45-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> use ApolloPortalDB;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
MariaDB [ApolloPortalDB]> show tables;
MariaDB [ApolloPortalDB]> select * from ServerConfig\G;
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> use ApolloPortalDB;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
(1)下载apollo-portal软件
[root@mfyxw50 ~]# cd /opt/src/
[root@mfyxw50 ~]# wgethttps://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-portal-1.5.1-github.zip
(2)解压apollo-portal软件至相对应的目录
[root@mfyxw50 ~]# mkdir /data/dockerfile/apollo-portal
[root@mfyxw50 ~]# unzip /opt/src/apollo-portal-1.5.1-github.zip -d /data/dockerfile/apollo-portal
(3)删除apollo-portal目录用不到的文件
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/apollo-portal-1.5.1-sources.jar
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/apollo-portal.conf
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/scripts/shutdown.sh
(4)配置数据库连接串
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/config/application-github.properties << EOF
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
EOF
(5)配置Portal的mate service(此处可以不修改，在configmap处已经引入了)
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/config/apollo-env.properties << EOF
local.meta=http://localhost:8080
dev.meta=http://config.od.com:8080
fat.meta=http://fill-in-fat-meta-server:8080
uat.meta=http://fill-in-uat-meta-server:8080
lpt.meta=${lpt_meta}
pro.meta=http://fill-in-pro-meta-server:8080
EOF
(6)更新apollo-portal的startup.sh文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/scripts/startup.sh << EOF
#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=\$(hostname -i)
# SERVER_URL="http://localhost:\$SERVER_PORT"
SERVER_URL="http://\${APOLLO_PORTAL_SERVICE_NAME}:\${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="\$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="\$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="\$JAVA_OPTS -Dserver.port=\$SERVER_PORT -Dlogging.file=\$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=\$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "\$JAVA_HOME" ]] && [[ -x "\$JAVA_HOME/bin/java" ]]; then
javaexe="\$JAVA_HOME/bin/java"
elif
type -p java > /dev/null 2>&1; then
javaexe=\$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo
"Unable to find Java"
exit 1
fi
if [[ "\$javaexe" ]]; then
version=\$("\$javaexe" -version 2>&1 | awk -F '"'
'/version/ {print \$2}')
version=\$(echo
"\$version" | awk -F. '{printf("%03d%03d",\$1,\$2);}')
# now version is of format 009003 (9.3.x)
if [ \$version -ge 011000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 010000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 009000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="\$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="\$JAVA_OPTS -Xloggc:\$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="\$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf
"\$(date) ==== Starting ==== \n"
cd \`dirname \$0\`/..
chmod 755 \$SERVICE_NAME".jar"
./\$SERVICE_NAME".jar" start
rc=\$?;
if [[ \$rc != 0 ]];
then
echo
"\$(date) Failed to start \$SERVICE_NAME.jar, return code: \$rc"
exit \$rc;
fi
tail -f /dev/null
EOF
在运维主机(mfyxw50.mfyxw.com)上操作
(1)编写Apollo-Portal的Docker文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/Dockerfile << EOF
FROM harbor.od.com/base/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo
"Asia/Shanghai" > /etc/timezone
ADD apollo-portal-\${VERSION}.jar /apollo-portal/apollo-portal.jar
ADD config/ /apollo-portal/config
ADD scripts/ /apollo-portal/scripts
CMD ["/apollo-portal/scripts/startup.sh"]
EOF
(2)制作Apollo-portal镜像
[root@mfyxw50 ~]# cd /data/dockerfile/apollo-portal
[root@mfyxw50 apollo-portal]# docker build . -t harbor.od.com/infra/apollo-portal:v1.5.1
[root@mfyxw50 apollo-portal]# docker login harbor.od.com
[root@mfyxw50 apollo-portal]# docker push harbor.od.com/infra/apollo-portal:v1.5.1
在mfyxw10.mfyxw.com主机操作
(1)设置域名
[root@mfyxw10 ~]# cat > /var/named/od.com.zone << EOF
\$ORIGIN od.com.
\$TTL 600 ; 10 minutes
@ IN SOA dns.od.com. dnsadmin.od.com. (
;序号请加1，表示比之前版本要新
2020031312 ; serial
10800 ; refresh (3 hours)
900 ; retry (15 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS dns.od.com.
\$TTL 60 ; 1 minute
dns A 192.168.80.10
harbor A 192.168.80.50 ;添加harbor记录
k8s-yaml A 192.168.80.50
traefik A 192.168.80.100
dashboard A 192.168.80.100
zk1 A 192.168.80.10
zk2 A 192.168.80.20
zk3 A 192.168.80.30
jenkins A 192.168.80.100
dubbo-monitor A 192.168.80.100
demo A 192.168.80.100
mysql A 192.168.80.10
config A 192.168.80.100
portal A 192.168.80.100
EOF
(2)重启DNS服务
[root@mfyxw10 ~]# systemctl restart named
(3)测试解析
[root@mfyxw10 ~]# dig
-t
A
portal.od.com @192.168.80.10 +short
192.168.80.100
在运维主机mfyxw50.mfyxw.com上操作
(1)创建用于存放Apollo-portal资源配置文件目录
[root@mfyxw50 ~]# mkdir -p /data/k8s-yaml/apollo-portal
(2)deployment.yaml文件内容如下：
[root@mfyxw50
~]#
cat
> /data/k8s-yaml/apollo-portal/deployment.yaml << EOF
kind:
Deployment
apiVersion:
extensions/v1beta1
metadata:
name:
apollo-portal
namespace:
infra
labels:
name:
apollo-portal
spec:
replicas:
1
selector:
matchLabels:
name:
apollo-portal
template:
metadata:
labels:
app:
apollo-portal
name:
apollo-portal
spec:
volumes:
- name:
configmap-volume
configMap:
name:
apollo-portal-cm
containers:
- name:
apollo-portal
image:
harbor.od.com/infra/apollo-portal:v1.5.1
ports:
- containerPort:
8080
protocol:
TCP
volumeMounts:
- name:
configmap-volume
mountPath:
/apollo-portal/config
terminationMessagePath:
/dev/termination-log
terminationMessagePolicy:
File
imagePullPolicy:
IfNotPresent
imagePullSecrets:
- name:
harbor
restartPolicy:
Always
terminationGracePeriodSeconds:
30
securityContext:
runAsUser:
0
schedulerName:
default-scheduler
strategy:
type:
RollingUpdate
rollingUpdate:
maxUnavailable:
1
maxSurge:
1
revisionHistoryLimit:
7
progressDeadlineSeconds:
600
EOF
(3)service.yaml文件内容如下：
[root@mfyxw50
~]#
cat
> /data/k8s-yaml/apollo-portal/service.yaml << EOF
kind:
Service
apiVersion:
v1
metadata:
name:
apollo-portal
namespace:
infra
spec:
ports:
- protocol:
TCP
port:
8080
targetPort:
8080
selector:
app:
apollo-portal
clusterIP:
None
type:
ClusterIP
sessionAffinity:
None
EOF
(4)Ingress.yaml文件内容如下：
[root@mfyxw50
~]#
cat
> /data/k8s-yaml/apollo-portal/Ingress.yaml << EOF
kind:
Ingress
apiVersion:
extensions/v1beta1
metadata:
name:
apollo-portal
namespace:
infra
spec:
rules:
- host:
portal.od.com
http:
paths:
- path:
/
backend:
serviceName:
apollo-portal
servicePort:
8080
EOF
(5)ConfigMap.yaml文件内容如下：
[root@mfyxw50
~]#
cat
> /data/k8s-yaml/apollo-portal/configmap.yaml << EOF
apiVersion:
v1
kind:
ConfigMap
metadata:
name:
apollo-portal-cm
namespace:
infra
data:
application-github.properties:
|
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
app.properties: |
appId=100003173
apollo-env.properties: |
dev.meta=http://config.od.com
EOF
在master节点(mfyxw30.mfyxw.com或mfyxw40.mfyxw.com)任意一台上执行即可
(1)应用Apollo-portal资源配置清单
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/deployment.yaml
deployment.extensions/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/service.yaml
service/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/Ingress.yaml
ingress.extensions/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/configmap.yaml
configmap/apollo-portal-cm created
(2)查看Apollo-portal的pod是否正常启动
[root@mfyxw30 ~]# kubectl get pod -n infra
NAME READY STATUS RESTARTS AGE
apollo-adminservice-5cccf97c64-bhqzb 1/1 Running 1
6h22m
apollo-configservice-5f6555448-7wxsp 1/1 Running 1
6h16m
apollo-portal-57bc86966d-jz5vg 1/1 Running 0
112s
dubbo-monitor-6676dd74cc-9hghb 1/1 Running 13
17d
dubbo-monitor-6676dd74cc-rd86g 1/1 Running 12
17d
jenkins-b99776c69-p6skp 1/1 Running 17
39d
开源软件都要改默认密码
Apollo-portal默认的用户为：apollo 密码为：admin 第一次登录后，强烈建议修改密码，在此修改密码为：admin123
在web界面修改数据库数据
对应修改了数据库键值如下
管理员工具-系统信息
七、实战配置dubbo微服务接收apollo配置中心管理
创建项目
新增配置
添加配置
#zk1是测试环境
发布配置
Jenkins-重新制作dubbo-service的harbor镜像
#apollo分支编写的参数变量是对应apollo注册的参数
Jenkins-编写拉取参数
修改dubbo-service的dp资源配置清单
cat dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: app
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_200805_1600
#新构建的镜像地址
ports:
- containerPort: 20880
protocol: TCP
env:
- name: JAR_BALL
value: dubbo-server.jar
- name: C_OPTS #添加的新环境变量
value: -Denv=dev -Dapollo.meta=http://config.od.com #添加的新容器启动参数
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
小结-apollo配置中心如何工作的
对应了dubbo-service服务提供者程序中如下2个变量
Register地址代表,服务注册的zk数据库地址,
Port代表,monitor探查dubbo-service服务状态暴露的端口,如果这里改了(dubbo-monitor应当也该改地址)
#相当于以环境变量的方式,在程序启动时中调用.
在Portal的web界面创建项目,写入键值,相当于在dashboard修改了configmap中的配置
Portal会把数据传递给adminservice,adminservice又把数据存到configdb中
在pod的dp中指定获取数据路径.指定config.od.com.当删除重建pod时会在这个地址读取定义的变量数据
注: 可以在程序中实现apollo监听器的方法,实现实时监听apollo配置中心键值变化,实时reload生效,达到pod更新配置的效果
不需要,人为删除pod重启
八、apollo配置中心创建
和dubbo服务提供者一样.重新制作dubbo-web的harbor镜像(将配置环境变量化)
gitee-消费者代码与apollo配置中心连接点
创建apollo配置中心
提交,再点一下发布
#开始构建
修改dubbo-demo-消费者dp资源清单
再次应用.apollo的配置就在pod里的项目生效了
九、实战使用apollo配置中心管理测试环境和生产环境
在实际企业环境,应该将测试环境和生产环境放到2个不同的k8s集群,而不是现在这样以名称空间区分
环境
命名空间
应用
测试环境（TEST）
test
apollo-config，apollo-admin,zk1,apolloconfigdb
测试环境（TEST）
test
dubbo-demo-service，dubbo-demo-web
生产环境（PROD）
prod
apollo-config，apollo-admin, apolloconfigdb
生产环境（PROD）
prod
dubbo-demo-service，dubbo-demo-web,zk2
ops环境（infra）--共用
infra
jenkins，dubbo-monitor，apollo-portal
8.3.1: 环境准备
创建名称空间和secret
1)首先停掉apollo3兄弟
创建test名称空间
[root@hdss7-21 ~]# kubectl create ns test
namespace/test created
#在test名称空间创建secret资源
[root@hdss7-21 ~]# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=123456 -n test
11主机创建apolloconfigdb测试库
#修改apolloconfig.sql
[root@hdss7-11 ~]# vi apolloconfig.sql
[root@hdss7-11 ~]# mysql -uroot -p <apolloconfig.sql
查看创建的apolloconfig测试库
**#修改数据库连接eruke信息,
**
#测试库使用的地址改一下
MariaDB [ApolloConfigTestDB]> select * from ServerConfig\G;
MariaDB
[(none)]>
update
ApolloConfigTestDB.ServerConfig
set
ServerConfig.Value="http://config-test.od.com/eureka"
where
ServerConfig.Key="eureka.service.url";
**#添加数据库访问用户授权
**
MariaDB [(none)]> grant SELECT,DELETE,UPDATE,INSERT on ApolloConfigTestDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
Query OK, 0 rows affected (0.00 sec)
创建测试环境资源清单目录
[root@hdss7-200 k8s-yaml]# mkdir -pv test/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer}
8.3.2: 测试环境发布apollo-configservice
将资源清单依次拷过来修改
[root@hdss7-200 apollo-configservice]# pwd
/data/k8s-yaml/test/apollo-configservice
[root@hdss7-200 apollo-configservice]# cp /data/k8s-yaml/apollo-configservice/* .
[root@hdss7-200 apollo-configservice]# ls
configmap.yaml deployment.yaml Ingress.yaml service.yaml
**修改资源清单中(所有)
**
1.名称空间改为test
2.修改config.od.com地址为config-test.od.com
3. spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigTestDB?characterEncoding
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/configmap.yaml
kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/deployment.yaml
kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/service.yaml
kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/Ingress.yaml
8.3.3: 在测试环境发布apollo-adminservice
修改资源清单(configmap,dp)
**名称空间改成test
**
**Configmap改:
**
eureka.service.url = http://config-test.od.com/eureka
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigTestDB?characterEncoding=utf8
**应用资源配置清单:
**
kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/configmap.yaml
kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/deployment.yaml
8.3.4: 在测试环境发布dubbo-service
[root@hdss7-200 dubbo-demo-service]# pwd
/data/k8s-yaml/test/dubbo-demo-service
#修改资源清单
[root@hdss7-200 dubbo-demo-service]# vi dp.yaml
namespace: test
value: -Denv=fat -Dapollo.meta=http://config-test.od.com
#应用资源清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-service/dp.yaml
8.3.5: 在测试环境发布dubbo-cosumer
[root@hdss7-200 dubbo-demo-consumer]# cp /data/k8s-yaml/dubbo-consumer/*.yaml .
修改资源清单
namespace: test (ingress,dp,svc)
value: -Denv=dev -Dapollo.meta=http://config-test.od.com(dp)
- host: demo-test.od.com(ingress)
添加dns解析
demo-test A 10.4.7.10
[root@hdss7-11 ~]# systemctl restart named
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/dp.yaml
kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/svc.yaml
kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/ingress.yaml
测试访问
Demo-test.od.com/hello?name=xiao
8.4.1: 环境准备
创建名称空间和secret
**1)首先停掉apollo3兄弟
**
创建prod名称空间
[root@hdss7-21 ~]# kubectl create ns prod
namespace/test created
#在prod名称空间创建secret资源
[root@hdss7-21 ~]# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=123456 -n prod
11主机创建apolloconfigdb生产库
**创建生产环境的configdb
**
[root@hdss7-11 ~]# mysql -uroot -p <apolloconfig.sql
**#修改数据库连接eruke信息,
**
#生产库使用的地址改一下
MariaDB [ApolloConfigTestDB]> select * from ServerConfig\G;
MariaDB
[(none)]>
update
ApolloConfigProdDB.ServerConfig
set
ServerConfig.Value="http://config-prod.od.com/eureka"
where
ServerConfig.Key="eureka.service.url";
**#添加数据库访问用户授权
**
MariaDB [(none)]> grant SELECT,DELETE,UPDATE,INSERT on ApolloConfigProdDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
Query OK, 0 rows affected (0.00 sec)
创建生产环境资源清单目录
[root@hdss7-200 k8s-yaml]# mkdir -pv prod/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer}
8.4.2: 生产环境发布apollo-configservice
将资源清单依次拷过来修改
[root@hdss7-200 apollo-configservice]# pwd
/data/k8s-yaml/prod/apollo-configservice
[root@hdss7-200 apollo-configservice]# cp /data/k8s-yaml/test/apollo-configservice/* .
[root@hdss7-200 apollo-configservice]# sed -i 's/test/prod/g' ./* #替换成prod
再将连接的数据库地址改掉
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/configmap.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/deployment.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/service.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/Ingress.yaml
8.4.3: 在生产环境发布apollo-adminservice
修改资源清单(configmap,dp)
名称空间改成prod
**Configmap改:
**
eureka.service.url = http://config-prod.od.com/eureka
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigProdDB?characterEncoding=utf8
[root@hdss7-200 apollo-adminservice]# cp /data/k8s-yaml/test/apollo-adminservice/* .
[root@hdss7-200 apollo-adminservice]# sed -i 's/test/prod/g' ./* #不要这样改,会把harbor地址改错
手动修改数据库地址,域名
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/configmap.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/deployment.yaml
8.3.4: 在生产环境发布dubbo-service
添加dns解析
demo-prod A 10.4.7.10
[root@hdss7-11 ~]# systemctl restart named
修改资源配置清单
[root@hdss7-200 dubbo-demo-service]# cp /data/k8s-yaml/test/dubbo-demo-service/*.yaml .
namespace: prod
value: -Denv=pro -Dapollo.meta=http://apollo-configservice:8080 #此地址对应config-prod.od.com(apollo的service地址)
应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-service/dp.yaml
8.3.5: 在生产环境发布dubbo-consumer
[root@hdss7-200 dubbo-demo-consumer]# cp /data/k8s-yaml/test/dubbo-demo-consumer/*.yaml .
修改资源配置清单
名称空间
namespace: prod (ingress,dp,svc)
value: -Denv=pro -Dapollo.meta=http://config-prod.od.com
- host: demo-prod.od.com(ingress)
应用资源配置清单
kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/dp.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/svc.yaml
kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/ingress.yaml
测试访问
Demo-prod.od.com/hello?name=xiao
删除ApolloConfigDB
11主机上,mysql
drop database ApolloConfigDB;
修改portaldb可支持的环境列表
MariaDB [ApolloPortalDB]> use ApolloPortalDB;
MariaDB [ApolloPortalDB]> update ServerConfig set value='fat,pro' where Id=1; #可支持的环境列表修改
修改apollo-portal的cm资源清单
[root@hdss7-200 apollo-portal]# vi /data/k8s-yaml/apollo-portal/configmap.yaml
修改标黄2行
**应用cm资源清单
**
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/configmap.yaml
添加dns解析(apollo-service)
删除apollo-portaldb中的dev项目数据
MariaDB [ApolloPortalDB]> use ApolloPortalDB;
MariaDB [ApolloPortalDB]> truncate table AppNamespace;
MariaDB [ApolloPortalDB]> truncate table App;
重新启动portaldb
Apollo配置中心配置微服务参数—dubbo服务提供者
登录apollo-potral
系统参数---查询apollo.portal.envs
配置apollo-连接dubbo微服务
#检查admin service资源清单,发现harbor地址被替换错了
#如果apollo开启之前,项目创建有问题,需要删除重建
添加测试环境参数
#添加对应环境的参数,添加完后点发布再编辑生产环境
#生产环境参数发布
Apollo配置中心配置微服务参数—dubbo服务消费者
创建项目
Gitee提交消费者更改代码
jenkins发布
修改测试环境消费者dp资源配置文件使用新版镜像
测试环境测试新版镜像,如果正常可以访问
再将生产环境消费者dp资源配置文件使用新版镜像(用测试环境做的镜像包)
故障排查
1. dubbo的3个组件,提供者,消费者,发现者,他们通信的端口,zk地址需要保证一致性
2. dubbo的3个组件的启动顺序需要注意,monitor需要在后启动,不然在web界面容易发现不了其他2个组件
3.apollo记得把之前的apolloconfigdb删掉,其他配置要记得检查
4. dubbo-monitor网站打不开,一定是zookeeper宕了.
5. 所有的问题都排查过了,发现dashboard上删除掉的dp,在k8s上居然没被删掉
6.启动顺序,zk,portal,数据库,apolloservice,apolloadmin,dubboservice,dubboconsumer
7.最好使用kubectl来初始启动集群
手机扫一扫
移动阅读更方便
你可能感兴趣的文章