且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

删除zookeeper中的kafka消费者组

更新时间:2023-01-19 08:57:45

目前,据我所知,删除 Kafka 消费者组的唯一方法是手动删除 Zookeeper 路径 /consumers/[group_id].

Currently, as I know, the only way to remove a Kafka consumer group is manually deleting Zookeeper path /consumers/[group_id].

如果你只是想删除一个消费者组,手动删除Zookeeper路径没有什么可担心的,但是如果你是为了rewinding offsets而做的,下面的会很有帮助.

If you just want to delete a consumer group, there is nothing to worry about manually deleting the Zookeeper path, but if you do it for rewinding offsets, the below will be helpful.

首先,在移除Zookeeper路径之前,您应该停止所有属于该消费者组的消费者.如果不这样做,这些消费者将不会使用新生成的消息,并且很快就会关闭与 Zookeeper 集群的连接.

First of all, you should stop all the consumers belongs to the consumer group before removing the Zookeeper path. If you don't, those consumers will not consume newly produced messages and will soon close connections to the Zookeeper cluster.

当您重新启动消费者时,如果您希望消费者从头开始,请将 auto.offset.reset 属性赋予 smallest(或 earliest 在新的 Kafka 版本中).该属性的默认值是 largest(或新 Kafka 版本中的 latest),这使您重新启动的消费者在最大偏移后读取,而这反过来又只消耗新生成的消息.有关该属性的更多信息,请参阅 Kafka 文档中的 消费者配置.

When you restart the consumers, if you want the consumers to start off from the beginning, give auto.offset.reset property to smallest (or earliest in new Kafka releases). The default value of the property is largest (or latest in new Kafka releases) which makes your restarting consumers read after the largest offset which in turn consuming only newly produced messages. For more information about the property, refer to Consumer Config in the Kafka documentation.

仅供参考,有一个问题如何在消费者中回滚偏移量? 在 Kafka FAQ 中,但它给我的帮助不大.

FYI, there is a question How can I rewind the offset in the consumer? in Kafka FAQ, but it gave me not much help.