Keepalived介绍
Keepalived的重要功能
Keepalived如何故障切换实现高可用?
(1) Keepalived高可用服务之间的故障切换转移,是通过VRRP(Virtual Router Redundancy Protocol,虚拟路由器冗余协议)来实现的。
(2) 在Keepalived服务正常工作时,主Master节点会不断地向备节点发送(多播的方式)心跳消息,用以告诉备Backup节点自己还活着,当主Master节点发生故障时,就无法发送心跳消息,备节点也就因此无法继续检测到来自主Master节点的心跳了,于是调用自身的接管程序,接管主Master节点的IP资源及服务。而当主Master节点恢复时,备Backup节点又会释放主节点故障时自身接管的IP资源及服务,恢复到原来的备用角色。
Keepalived工作原理
Keepalived的组件图
上图是Keepalived的功能体系结构,大致分两层:用户空间(user space)和内核空间(kernel space)。
Keepalived的所有功能是配置keepalived.conf文件来实现的。
Keepalived正常启动的时候,共启动3个进程
一个是父进程,负责监控其子进程;一个是VRRP子进程,另外一个是checkers子进程;两个子进程都被系统watchlog看管,两个子进程各自负责复杂自己的事。
Healthcheck子进程检查各自服务器的健康状况,,例如http,lvs。如果healthchecks进程检查到master上服务不可用了,就会通知本机上的VRRP子进程,让他删除通告,并且去掉虚拟IP,转换为BACKUP状态。
注意:keepalived和LVS完全是两码事,只不过他们各负其责相互配合而已。
Keepalived高可用服务器的“裂脑”问题——什么是裂脑。
导致裂脑发生的原因,有以下几种原因:
提示:Keepalived配置里同一VRRP实例如果virtual_router_id两端参数配置不一致,也会导致裂脑问题发生。
解决裂脑的常见方案。
下面是生产场景检测裂脑故障的一些思路:
(1)简单判断的思想:只要备节点出现VIP就报警,这个报警有两种情况,一是主机宕机了备机接管了;二是主机没宕,裂脑了。不管属于哪个情况,都进行报警,然后由人工查看判断及解决。
(2)比较严谨的判断:备节点出现对应VIP,并且主节点及对应服务(如果能远程连接主节点看是否有VIP就更好了)还活着,就说明发生裂脑了。
准备环境
属性
KeepAlived MASTER
KeepAlived BACKUP
节点
KeepAlived-Master
KeepAlived-Backup
系统
CentOS Linux release 7.5.1804 (Minimal)
CentOS Linux release 7.5.1804 (Minimal)
内核
3.10.0-862.el7.x86_64
3.10.0-862.el7.x86_64
SELinux
setenforce 0 | disabled
setenforce 0 | disabled
Firewlld
systemctl stop/disable firewalld
systemctl stop/disable firewalld
IP地址
172.16.70.37
172.16.70.4
# 修改主机名
[root@locahost ~]# hostnamectl set-hostname --static KeepAlived-Master && exec bash
[root@KeepAlived-Master ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
[root@KeepAlived-Master ~]# uname -r
3.10.0-862.el7.x86_64
[root@KeepAlived-Master ~]# sed -i '7s/enforcing/disabled/' /etc/selinux/config
[root@KeepAlived-Master ~]# setenforce 0
[root@KeepAlived-Master ~]# systemctl stop firewalld && systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@KeepAlived-Master ~]# hostname -I
172.16.70.37
[root@KeepAlived-Master ~]# yum list | grep keepalived
keepalived.x86_64 1.3.5-19.el7 base
[root@KeepAlived-Master ~]# yum install -y vim net-tools wget curl lrzsz lsof
[root@KeepAlived-Master ~]# yum install -y openssl openssl-devel libnl libnl-devel gcc
[root@KeepAlived-Master ~]# wget www.keepalived.org/software/keepalived-2.0.20.tar.gz
[root@KeepAlived-Master ~]# md5sum keepalived-2.0.20.tar.gz
a5966e8433b60998709c4a922a407bac keepalived-2.0.20.tar.gz
[root@KeepAlived-Master ~]# tar -xf keepalived-2.0.20.tar.gz
[root@KeepAlived-Master ~]# cd keepalived-2.0.20
[root@KeepAlived-Master keepalived-2.0.20]# ./configure --help | less # 查看选择适合的编译参数
[root@KeepAlived-Master keepalived-2.0.20]# ./configure --prefix=/usr/local/app/keepalived --with-systemdsystemunitdir=/usr/local/app/keepalived --enable-log-file
……
….最后正常编译完成后如下
Keepalived version : 2.0.20
Compiler : gcc
Preprocessor flags : -D_GNU_SOURCE
……
……
Linker flags : -pie -Wl,-z,relro -Wl,-z,now
Extra Lib : -lm -lcrypto -lssl -lnl
Use IPVS Framework : Yes
IPVS use libnl : Yes
IPVS syncd attributes : No
IPVS 64 bit stats : No
HTTP_GET regex support : No
fwmark socket support : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
Use VRRP authentication : Yes
With ip rules/routes : Yes
With track_process : Yes
With linkbeat : Yes
Use BFD Framework : No
SNMP vrrp support : No
SNMP checker support : No
SNMP RFCv2 support : No
SNMP RFCv3 support : No
DBUS support : No
SHA1 support : No
Use JSON output : No
libnl version : 1
Use IPv4 devconf : No
Use iptables : Yes
Use libiptc : No
Use libipset : No
Use nftables : No
init type : systemd
Strict config checks : No
Build genhash : Yes
Build documentation : No
[root@KeepAlived-Master keepalived-2.0.20]# make -j 4 && make install
[root@KeepAlived-Master ~]# ls /usr/local/app/keepalived/
bin etc keepalived.service sbin share
[root@KeepAlived-Master ~]# ln -s /usr/local/app/keepalived/sbin/keepalived /usr/local/sbin/
[root@KeepAlived-Master ~]# ls -l /usr/sbin/keepalived
lrwxrwxrwx. 1 root root 42 Sep 18 14:31 /usr/sbin/keepalived -> /usr/local/apps/keepalived/sbin/keepalived
[root@KeepAlived-Master ~]# keepalived -v
Keepalived v2.0.20 (01/22,2020)
Copyright(C) 2001-2020 Alexandre Cassen, acassen@gmail.com
Built with kernel headers for Linux 3.10.0
Running on Linux 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018
configure options: --prefix=/usr/local/app/keepalived --with-systemdsystemunitdir=/usr/local/app/keepalived --enable-log-file
Config options: LVS VRRP VRRP_AUTH OLD_CHKSUM_COMPAT FIB_ROUTING FILE_LOGGING LOG_FILE_APPEND
System options: PIPE2 SIGNALFD INOTIFY_INIT1 VSYSLOG …… INET6_ADDR_GEN_MODE SO_MARK SCHED_RESET_ON_FORK
[root@KeepAlived-Master ~]# keepalived -h
Usage: keepalived [OPTION…]
-f, --use-file=FILE Use the specified configuration file
-P, --vrrp Only run with VRRP subsystem
-C, --check Only run with Health-checker subsystem
--all Force all child processes to run, even if have no configuration
-l, --log-console Log messages to local console
-D, --log-detail Detailed log messages
-S, --log-facility=[0-7] Set syslog facility to LOG_LOCAL[0-7]
-G, --no-syslog Don't log via syslog
-u, --umask=MASK umask for file creation (in numeric form)
-X, --release-vips Drop VIP on transition from signal.
-V, --dont-release-vrrp Don't remove VRRP VIPs and VROUTEs on daemon stop
-I, --dont-release-ipvs Don't remove IPVS topology on daemon stop
-R, --dont-respawn Don't respawn child processes
-n, --dont-fork Don't fork the daemon process
-d, --dump-conf Dump the configuration data
-p, --pid=FILE Use specified pidfile for parent process
-r, --vrrp_pid=FILE Use specified pidfile for VRRP child process
-c, --checkers_pid=FILE Use specified pidfile for checkers child process
-a, --address-monitoring Report all address additions/deletions notified via netlink
-s, --namespace=NAME Run in network namespace NAME (overrides config)
-m, --core-dump Produce core dump if terminate abnormally
-M, --core-dump-pattern=PATN Also set /proc/sys/kernel/core_pattern to PATN (default 'core')
-i, --config-id id Skip any configuration lines beginning '@' that don't match id
or any lines beginning @^ that do match.
The config-id defaults to the node name if option not used
--signum=SIGFUNC Return signal number for STOP, RELOAD, DATA, STATS
-t, --config-test[=LOG_FILE] Check the configuration for obvious errors, output to
stderr by default
-v, --version Display the version number
-h, --help Display this help message
[root@KeepAlived-Master ~]# cd /usr/local/app/keepalived/
[root@KeepAlived-Master keepalived]# ls
bin etc keepalived.service logs run sbin share
[root@KeepAlived-Master keepalived]# cp etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
[root@KeepAlived-Master keepalived]# diff etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
21c21
interface eth0
23c23
< priority 110 # 修改此行
priority 100
[root@KeepAlived-Master keepalived]# diff keepalived.conf keepalived.conf_BACKUP
20c20
state BACKUP
23c23
< priority 110
priority 100
[root@KeepAlived-Master ~]# vim /usr/lib/systemd/system/keepalived.service
[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target
Wants=network-online.target
[Service]
Type=forking
PIDFile=/run/keepalived.pid # 自定义项
KillMode=process
EnvironmentFile=-/usr/local/app/keepalived/etc/sysconfig/keepalived # 自定义项
ExecStart=/usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf $KEEPALIVED_OPTIONS # 自定义项
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target
[root@KeepAlived-Master ~]# systemctl daemon-reload
[root@KeepAlived-Master ~]# systemctl start|stop|restart keepalived
[root@KeepAlived-Master ~]# ps -ef| grep keepalived
root 8235 1 0 15:37 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8236 8235 1 15:37 ? 00:00:03 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8237 8235 0 15:37 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8249 1149 0 15:41 pts/0 00:00:00 grep --color=auto keepalived
[root@KeepAlived-Master ~]# ip addr | grep '192.168.200'
inet 192.168.200.16/32 scope global ens33
inet 192.168.200.17/32 scope global ens33
inet 192.168.200.18/32 scope global ens33
[root@KeepAlived-Master ~]# systemctl enable keepalived
[root@KeepAlived-Master ~]# systemctl list-unit-files | grep keepalived
keepalived.service enabled
[root@KeepAlived-Master ~]# vim /usr/local/app/keepalived/etc/sysconfig/keepalived
…..修为如下
KEEPALIVED_OPTIONS="-D -S 0"
[root@KeepAlived-Master ~]# echo "local0.* /usr/local/app/keepalived/logs/keepalived.log" >> /etc/rsyslog.conf
[root@KeepAlived-Master ~]# systemctl restart rsyslog
第二部分:故障转移测试
# 1.Master机操作
[root@KeepAlived-Master ~]# systemctl restart keepalived
[root@KeepAlived-Master ~]# ps -ef |grep keepalived
root 8548 1 0 17:17 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8549 8548 0 17:17 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8550 8548 0 17:17 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D -S 0
root 8552 1149 0 17:18 pts/0 00:00:00 grep --color=auto keepalived
[root@KeepAlived-Master ~]# ip addr | grep '192.168.200' # 查看vip资源情况 (默认vip是在主节点上的)
inet 192.168.200.16/32 scope global ens33
inet 192.168.200.17/32 scope global ens33
inet 192.168.200.18/32 scope global ens33
[root@KeepAlived-Backup ~]# systemctl restart keepalived
[root@KeepAlived-Backup ~]# ps -ef |grep keepalived
root 8345 1 0 Sep23 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 8346 8345 0 Sep23 ? 00:00:33 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 8347 8345 0 Sep23 ? 00:00:14 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 14260 1149 0 15:12 pts/0 00:00:00 grep --color=auto keepalived
[root@KeepAlived-Backup ~]# ip addr | grep '192.168.200' # 从节点没有vip资源
[root@KeepAlived-Master ~]# systemctl stop keepalived
[root@KeepAlived-Master ~]# ps -ef |grep keepalived
root 14488 1149 0 15:18 pts/0 00:00:00 grep --color=auto keepalived
[root@KeepAlived-Master ~]# ip addr | grep '192.168.200'
[root@KeepAlived-Backup ~]# ps -ef |grep keepalived
root 8345 1 0 Sep23 ? 00:00:00 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 8346 8345 0 Sep23 ? 00:00:33 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 8347 8345 0 Sep23 ? 00:00:14 /usr/local/app/keepalived/sbin/keepalived -f /usr/local/app/keepalived/etc/keepalived/keepalived.conf -D
root 14269 1149 0 15:20 pts/0 00:00:00 grep --color=auto keepalived
[root@KeepAlived-Backup ~]# ip addr | grep '192.168.200'
inet 192.168.200.16/32 scope global ens33
inet 192.168.200.17/32 scope global ens33
inet 192.168.200.18/32 scope global ens33
[root@KeepAlived-Master ~]# systemctl start keepalived
[root@KeepAlived-Master ~]# ip addr | grep '192.168.200'
inet 192.168.200.16/32 scope global ens33
inet 192.168.200.17/32 scope global ens33
inet 192.168.200.18/32 scope global ens33
[root@KeepAlived-Backup ~]# ip addr | grep '192.168.200'
注意:以上操作, keepalived仅仅实现了两台机器的vip的故障转移功能, 即实现了双机热备, 避免了单点故障.
[root@KeepAlived-Master keepalived]# cat etc/keepalived/keepalived.conf # 默认配置文件解析
! Configuration File for keepalived
global_defs { # 全局定义区块
notification_email { # 指定keepalived在发生事件时(比如切换)发送通知邮件的邮箱
acassen@firewall.loc # 设置报警邮件地址,可以设置多个,每行一个。 需开启本机的sendmail服务
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc # keepalived在发生诸如切换操作时需要发送email通知地址
smtp_server 192.168.200.1 # 指定发送email的smtp服务器
smtp_connect_timeout 30 # 设置连接smtp server的超时时间
router_id LVS_DEVEL # 运行keepalived的机器的一个标识,通常可设为hostname。故障发生时,发邮件时显示在邮件主题中的信息
vrrp_skip_check_adv_addr # 如果通告与接收的上一个通告来自相同master路由器,则跳过检查
vrrp_strict # 严格执行VRRP协议规范,此模式不支持节点单播
vrrp_garp_interval 0 # arp报文发送延迟
vrrp_gna_interval 0 # 消息发送延迟
}
vrrp_instance VI_1 { # vrrp实例定义区块
state MASTER # 指定keepalived的角色,MASTER表示此主机是主服务器,BACKUP表示此主机是备用服务器,并且需要大写这些字符
interface eth0 # 指定HA监测网络的接口。实例绑定的网卡,因为在配置虚拟IP的时候必须是在已有的网卡上添加
virtual_router_id 51 # 虚拟路由标识,这个标识是一个数字,同一个vrrp实例使用唯一的标识。即同一vrrp_instance下,MASTER和BACKUP必须是一致,否则将出现脑裂问题
priority 100 # 定义优先级,数字越大,优先级越高,在同一个vrrp_instance下,MASTER的优先级必须大于BACKUP的优先级
advert_int 1 # 设定MASTER与BACKUP负载均衡器之间同步检查的时间间隔,单位是秒
authentication { # 设置验证类型和密码。主从必须一样
auth_type PASS # 设置vrrp验证类型,主要有PASS和AH两种
auth_pass 1111 # 设置vrrp验证密码,在同一个vrrp_instance下,MASTER与BACKUP必须使用相同的密码才能正常通信
}
virtual_ipaddress { # VRRP HA 虚拟地址 如果有多个VIP,每个地址占一行,配置时最好明确指定子网掩码以及虚拟IP绑定的网络接口。否则,子网掩码默认是32位,绑定的接口和前面的interface参数配置的一致
192.168.200.16
192.168.200.17
192.168.200.18
}
}
virtual_server 192.168.200.100 443 { #设置virtual server: VIP:Vport
delay_loop 6 # service polling的delay时间,即服务轮询的时间间隔
lb_algo rr # LVS调度算法,rr|wrr|lc|wlc|lblc|sh|dh
lb_kind NAT # LVS集群模式,NAT|DR|TUN
persistence_timeout 50 # 会话保持时间(秒为单位),即以用户在120秒内被分配到同一个后端realserver
protocol TCP # 健康检查用的是TCP还是UDP
real\_server 192.168.201.100 443 { # 后端真实节点主机的权重等设置,主要,后端有几台这里就要设置几个
weight 1 # 给每台的权重,0表示失效(不知给他转发请求知道他恢复正常),默认是1
SSL\_GET { # 健康检查方式,HTTP\_GET|SSL\_GET|TCP\_CHECK|SMTP\_CHECK|MISC\_CHECK
url { # 要坚持的URL,可以有多个
path / # 具体路径
digest ff20ad2481f97b1754ef3e12ecd3a9cc
}
url {
path /mrtg/
digest 9b3a0c85a887a256d6939da88aabd8cd
}
connect\_timeout 3 # 连接超时时间
retry 3 # 重连次数
delay\_before\_retry 3 # 重连间隔
}
}
}
virtual_server 10.10.10.2 1358 {
delay_loop 6
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP
sorry\_server 192.168.200.200 1358
real\_server 192.168.200.2 1358 {
weight 1
HTTP\_GET {
url {
path /testurl/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl3/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
connect\_timeout 3
retry 3
delay\_before\_retry 3
}
}
real\_server 192.168.200.3 1358 {
weight 1
HTTP\_GET {
url {
path /testurl/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334c
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334c
}
connect\_timeout 3
retry 3
delay\_before\_retry 3
}
}
}
virtual_server 10.10.10.3 1358 {
delay_loop 3
lb_algo rr
lb_kind NAT
persistence_timeout 50
protocol TCP
real\_server 192.168.200.4 1358 {
weight 1
HTTP\_GET {
url {
path /testurl/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl3/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
connect\_timeout 3
retry 3
delay\_before\_retry 3
}
}
real\_server 192.168.200.5 1358 {
weight 1
HTTP\_GET {
url {
path /testurl/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl3/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
connect\_timeout 3
retry 3
delay\_before\_retry 3
}
}
}
至此,已简单实现Keepalived故障转移。
手机扫一扫
移动阅读更方便
你可能感兴趣的文章