问题描述
尝试理解Kafka中的一致性维护.请找出场景并帮助理解.
Try to understanding consistency maintenance in Kafka. Please find the scenario and help to understand.
Number of partition = 2
Replication factor = 3
Number of broker in the cluster = 4
在那种情况下，为了实现强一致性，应该有多少节点承认.ack = all 或 ack = 3 或任何其他值.请确认相同.
In that case, for achieving the strong consistency how many nodes should acknowledge. Either ack = all or ack = 3 or any other value. Please confirm for the same.
推荐答案
您可能有兴趣查看 当它绝对，肯定，必须在那里 来自卡夫卡峰会的谈话.
You might be interested in seeing When it Absolutely, Positively, Has to be There talk from Kafka Summit.
这是由 Cloudera 的一名工程师提供的，Cloudera 有他们的自己关于 Kafka 可用性的文档
Which was given by an engineer at Cloudera, and Cloudera has their own documenation on Kafka availability
总而言之，多于 1 个副本和多于 1 个同步副本是一个好的开始.然后在生产者上，如果您可以为数据可用性牺牲吞吐量，这意味着您必须在继续之前写入所有副本，然后 acks=all.否则，如果您相信 leader broker 是高可用的且不洁的 leader 选举是错误的，那么在大多数情况下 acks=1 应该没问题.
To summarize, more than 1 replica and higher than 1 in-sync replica is a good start. Then on the producer, if you are okay with sacrificing throughput for data availability, meaning you must have all replicas be written before continuing, then acks=all. Otherwise, if you trust the leader broker to be highly available with unclean leader election is false, then acks=1 should be okay in most cases.
acks=3 不是有效的配置.我认为您正在寻找复制因子为 3 的 min.insync.replicas=2 和 acks=all；来自上面的链接
acks=3 isn't a valid config, by the way. I think you are looking for min.insync.replicas=2 and acks=all with a replication factor of 3; from above link
如果 min.insync.replicas 设置为 2 并且 acks 设置为 all，则每条消息必须成功写入至少两个副本.这保证了消息不会丢失，除非两台主机都崩溃
此外，您可以启用事务性生产者，从 Kafka 0.11 开始，只处理一次
Also, you can enable the transactional producer, as of Kafka 0.11 to work towards exactly once processing
enable.idempotence=true
这篇关于Kafka如何实现强一致性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！