问题描述
Kafka版本:0.10.2.1，
Kafka Version : 0.10.2.1,
Kafka Producer error Expiring 10 record(s) for TOPIC:XXXXXX: 6686 ms has passed since batch creation plus linger time
org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s) for TOPIC:XXXXXX: 6686 ms has passed since batch creation plus linger time
推荐答案
之所以发生此异常，是因为您以比发送记录快得多的速率对记录进行排队.
This exception is occuring because you are queueing records at a much faster rate than they can be sent.
当调用send方法时，ProducerRecord将存储在内部缓冲区中，以发送给代理.一旦ProducerRecord被缓冲，该方法将立即返回，而不管其是否已发送.
When you call the send method, the ProducerRecord will be stored in an internal buffer for sending to the broker. The method returns immediately once the ProducerRecord has been buffered, regardless of whether it has been sent.
记录被分为几批发送给代理，以减少每条消息的传输窃听和增加吞吐量.
Records are grouped into batches for sending to the broker, to reduce the transport overheard per message and increase throughput.
一旦将记录添加到批处理中，发送该批处理就会有一定的时间限制，以确保已在指定的持续时间内发送了该批处理.这由生产者配置参数request.timeout.ms控制，默认值为30秒.
Once a record is added into a batch, there is a time limit for sending that batch to ensure that it has been sent within a specified duration. This is controlled by the Producer configuration parameter, request.timeout.ms, which defaults to 30 seconds.
如果批处理的排队时间超过超时限制，则将引发异常.该批次中的记录将从发送队列中删除.
If the batch has been queued longer than the timeout limit, the exception will be thrown. Records in that batch will be removed from the send queue.
生产者配置block.on.buffer.full，metadata.fetch.timeout.ms和timeout.ms已被删除.他们最初在Kafka 0.9.0.0中被弃用.
因此请尝试增加 request.timeout.ms
不过，如果您对吞吐量有任何疑问，也可以参考以下博客
Still, if you have any problem related to throughput, you can also refer following blog
这篇关于Kafka Producer错误TOPIC:XXXXXX的10条记录已过期:自创建批处理以来，已经过去了6686毫秒的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！