且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Kafka Streams - 如何扩展 Kafka 存储生成的变更日志主题

更新时间:2023-02-07 12:52:45

如果你想将同一个状态加载到多个实例中,你应该使用 GlobalKTable 和唯一的 application.id 覆盖所有实例(builder.globalTable()).

If you want to load the same state into multiple instances, you should use GlobalKTable and a unique application.id over all instances (builder.globalTable()).

如果您使用 KTable 数据被分区,迫使您为每个实例使用不同的 application.id.这可以被认为是一种反模式.

If you use KTable data is partitioned forcing you to use different application.id for each instance. This can be considered an anti-pattern.

我也不确定,为什么要执行 groupBy((k, v) -> KeyValue.pair(k, v)).reduce() -- 这会导致不必要的重新分区主题.

I am also not sure, why you do groupBy((k, v) -> KeyValue.pair(k, v)).reduce() -- this results in an unnecessary repartition topic.

对于 table() 操作符生成的变更日志主题,1.01.1 版本中存在一个已知错误,如果 StreamsBuilder 被使用(KStreamBuilder 不受影响).它在 2.0 版本(https://issues.apache.org/jira/browse/KAFKA-6729)

For the generated changelog topics for table() operator, there is a know bug in 1.0 and 1.1 release if StreamsBuilder is used (KStreamBuilder is not affected). Its fixed in 2.0 release (https://issues.apache.org/jira/browse/KAFKA-6729)