kafka

(Updated: )
技术
,

路径

1
brew list kafka

/usr/local/Cellar/kafka/2.8.0/

开启

  1. 在kafka目录直接启动
1
2
zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties &
kafka-server-start /usr/local/etc/kafka/server.properties &
  1. 通过brew services插件启动
1
2
brew services start zookeeper
brew services start kafka

停止

1.直接停止

1
kafka-server-stop config/server.properties
  1. 通过brew services插件停止
1
brew services stop kafka

查询topic

1
➜  2.8.0 kafka-topics --zookeeper localhost:2181 --list

新建topic

1
➜  2.8.0 kafka-topics --zookeeper localhost:2181 --create --topic test --partitions 10 --replication-factor 1

测试环境

kafka
路径 /usr/local/kafka_2.12-2.4.1
willing
路径 /home/willing/willing

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
kafka
1）查看当前服务器中的所有topic
bin/kafka-topics.sh --zookeeper localhost:2181/ --list
2）创建topic
bin/kafka-topics.sh --zookeeper localhost:2181/ --create --replication-factor 1 --partitions 3 --topic CLUE.FORWARD.SH2
选项说明：
--topic 定义topic名
--replication-factor  定义副本数
--partitions  定义分区数
3）删除topic
bin/kafka-topics.sh --zookeeper localhost:2181/ --delete --topic first
4）发送消息
bin/kafka-console-producer.sh --broker-list 172.17.16.10:9092 --topic first --consumer.config ./config/consumer.properties
5）消费消息
bin/kafka-console-consumer.sh  --bootstrap-server 172.17.16.10:9092 --topic REPORT.API.SH2  --from-beginning --consumer.config ./config/consumer.properties
--from-beginning：会把主题中现有的所有的数据都读取出来。
不指定。接着上次读
6）查看某个Topic的详情
bin/kafka-topics.sh --zookeeper localhost:2181/ --describe --topic first
7）修改分区数
bin/kafka-topics.sh --zookeeper localhost:2181/ --alter --topic first --partitions 6
8）启动kafka
bin/kafka-server-start-saal.sh -daemon config/server-sasl.properties
9)查看消费者组
export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/local/kafka_2.12-2.4.1/config/kafka_client_jaas.conf"
./kafka-consumer-groups.sh --bootstrap-server 172.17.16.10:9092 --command-config ../config/consumer.properties --describe --group 'test-consumer-group2'


bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic maxwell