闲来如事，突发兴趣倒腾一下了kafka。其实类似环境搭建的博客很多，我这里算是整合一下，另外写出自己的过程与看法。完整的过程如下：

1）先准备好3台服务器。我准备的机器hostname/ip为：

kafka/192.168.56.151 ,   kafka2/192.168.56.152,  kafka3/192.168.56.103

2) 搭建zookeeper集群。可以参考如下链接：

https://www.cnblogs.com/ysocean/p/9860529.html

搭建好，并配置好了环境变量后。可以分别启动3台机器，并查看集群状态。如下可以看到kafka机器为leader，kafka2 & kafka3为follower

[root@kafaka ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
[root@kafaka ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader

[root@kafaka3 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
[root@kafaka3 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@kafaka2 conf]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper … STARTED
[root@kafaka2 conf]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

3)安装配置kafka brokers。

kafka的安装过程，可以参考如下文档

https://blog.csdn.net/sunsijia21983/article/details/115305971

我在这里，配置了6个brokers。分别为broker0 ~ broker5。每台机器两个broker。

[root@kafaka config]# pwd
/usr/local/kafka_2.13-2.7.0/config
[root@kafaka config]# ls -l server.properties.*
-rw-r--r-- 1 root root 7070 Jul 9 23:46 server.properties.0
-rw-r--r-- 1 root root 7055 Jul 10 00:06 server.properties.1

server.properties.0的配置如下：

注意需要的是每个broker有不同的id。而且端口不能相同。还有就是log.dirs的目录需要先创建。zookeeper.connect参数为所有3台服务器的zookeeper地址

在kafka2 & kafka3的机器需要使用不同的broker id，端口可以用同样的9092 & 9093，IP需要修改成机器IP(机器名也是可以的，前提是你配置了/etc/hosts文件

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0

Hostname and port the broker will advertise to producers and consumers. If not set,

log.dirs=/usr/local/kafka_2.13-2.7.0/broker0

The address the socket server listens on. It will get the value returned from

java.net.InetAddress.getCanonicalHostName() if not configured.

FORMAT:

listeners = listener_name://host_name:port

EXAMPLE:

listeners = PLAINTEXT://your.host.name:9092

#listeners=PLAINTEXT://192.168.56.150:9092

Hostname and port the broker will advertise to producers and consumers. If not set,

it uses the value for "listeners" if configured. Otherwise, it will use the value

returned from java.net.InetAddress.getCanonicalHostName().

#advertised.listeners=PLAINTEXT://your.host.name:9092
advertised.listeners=PLAINTEXT://192.168.56.151:9092

The default number of log partitions per topic. More partitions allow greater

parallelism for consumption, but this will also result in more files across

the brokers.

num.partitions=3

Zookeeper connection string (see zookeeper docs for details).

This is a comma separated host:port pairs, each corresponding to a zk

server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".

You can also append an optional chroot string to the urls to specify the

root directory for all kafka znodes.

zookeeper.connect=192.168.56.151:2181,192.168.56.152:2181,192.168.56.103:2181

server.properties.1的配置如下：

broker.id=1

listeners=PLAINTEXT://192.168.56.151:9093

advertised.listeners=PLAINTEXT://192.168.56.151:9093

num.partitions=3
zookeeper.connect=192.168.56.151:2181,192.168.56.152:2181,192.168.56.103:2181

3)启动kafka(在kafka/bin目录下启动kafka进程）

kafka机器

[root@kafaka bin]# pwd
/usr/local/kafka_2.13-2.7.0/bin

1018 ./kafka-server-start.sh ../config/server.properties.0 &
1019 ./kafka-server-start.sh ../config/server.properties.1 &

kafka2机器

1025 ./kafka-server-start.sh ../config/server.properties.2 &
1026 ./kafka-server-start.sh ../config/server.properties.3 &

kafka3机器

1003 ./kafka-server-start.sh ../config/server.properties.4 &
1004 ./kafka-server-start.sh ../config/server.properties.5 &

4)连接任一台机器的zookeeper，查看broker ids。如下可以看到我有0-5，六个broker上线了。

[root@kafaka bin]# zkCli.sh -server localhost:2181
Connecting to localhost:2181
Welcome to ZooKeeper!
JLine support is enabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[admin, brokers, cluster, config, consumers, controller, controller_epoch, feature, isr_change_notification, latest_producer_id_block, log_dir_event_notification, zookeeper]
[zk: localhost:2181(CONNECTED) 1] ls /brokers
[ids, seqid, topics]
[zk: localhost:2181(CONNECTED) 2] ls /brokers/ids
[0, 1, 2, 3, 4, 5]
[zk: localhost:2181(CONNECTED) 3]

如此，我可以算是搭建成功了一个高可用的kafka集群了。下一篇文章，我再来讲一讲如何测试kafka集群。