提问人：Sybuser 提问时间：11/18/2023 更新时间：11/18/2023 访问量：36
Kafka 节点断开连接——这些高频重复错误是什么意思，如何解决？
Kafka node disconnected - what do these high-frequency repeated errors mean and how to solve them?
问：
我在Spring Boot应用程序中使用Kafka生产者。
@Component
public class MessageProducer {
@Value("${topic.name}")
private String topic;
@Autowired
private KafkaTemplate<Integer, String> kafkaTemplate;
public void produceMessage(String msg) {
kafkaTemplate.send(topic, msg);
}
}
我在Spring boot配置文件中配置了以下属性：application.properties
spring.kafka.bootstrap-servers=...
topic.name=...
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
spring.kafka.properties.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="..." password="...";
当我调用 produceMessage 时，日志会不断填充以下消息：
2023-11-17T17:09:39.032+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -5 disconnected.
2023-11-17T17:09:39.032+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 0 due to node -5 being disconnected (elapsed time since creation: 170ms, elapsed time since send: 170ms, request timeout: 30000ms)
2023-11-17T17:09:39.032+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-105.server:9093 (id: -5 rack: null) disconnected
2023-11-17T17:09:39.134+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -3 disconnected.
2023-11-17T17:09:39.135+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 1 due to node -3 being disconnected (elapsed time since creation: 270ms, elapsed time since send: 270ms, request timeout: 30000ms)
2023-11-17T17:09:39.135+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-103.server:9093 (id: -3 rack: null) disconnected
2023-11-17T17:09:39.301+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -9 disconnected.
2023-11-17T17:09:39.301+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 2 due to node -9 being disconnected (elapsed time since creation: 55ms, elapsed time since send: 55ms, request timeout: 30000ms)
2023-11-17T17:09:39.301+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-109.server:9093 (id: -9 rack: null) disconnected
2023-11-17T17:09:39.493+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -1 disconnected.
2023-11-17T17:09:39.493+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 3 due to node -1 being disconnected (elapsed time since creation: 84ms, elapsed time since send: 84ms, request timeout: 30000ms)
2023-11-17T17:09:39.493+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-101.server:9093 (id: -1 rack: null) disconnected
2023-11-17T17:09:39.494+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -7 disconnected.
2023-11-17T17:09:39.494+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 4 due to node -7 being disconnected (elapsed time since creation: 83ms, elapsed time since send: 83ms, request timeout: 30000ms)
2023-11-17T17:09:39.494+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-107.server:9093 (id: -7 rack: null) disconnected
2023-11-17T17:09:39.810+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -6 disconnected.
2023-11-17T17:09:39.810+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 6 due to node -6 being disconnected (elapsed time since creation: 204ms, elapsed time since send: 204ms, request timeout: 30000ms)
2023-11-17T17:09:39.810+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-106.server:9093 (id: -6 rack: null) disconnected
2023-11-17T17:09:39.811+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -2 disconnected.
2023-11-17T17:09:39.811+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 5 due to node -2 being disconnected (elapsed time since creation: 206ms, elapsed time since send: 206ms, request timeout: 30000ms)
2023-11-17T17:09:39.811+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-102.server:9093 (id: -2 rack: null) disconnected
2023-11-17T17:09:39.974+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -8 disconnected.
2023-11-17T17:09:39.975+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 7 due to node -8 being disconnected (elapsed time since creation: 52ms, elapsed time since send: 52ms, request timeout: 30000ms)
2023-11-17T17:09:39.975+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-108.server:9093 (id: -8 rack: null) disconnected
2023-11-17T17:09:40.137+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -4 disconnected.
2023-11-17T17:09:40.137+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 8 due to node -4 being disconnected (elapsed time since creation: 54ms, elapsed time since send: 54ms, request timeout: 30000ms)
2023-11-17T17:09:40.137+01:00 WARN 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Bootstrap broker broker-104.server:9093 (id: -4 rack: null) disconnected
2023-11-17T17:09:40.304+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Node -5 disconnected.
2023-11-17T17:09:40.304+01:00 INFO 35184 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Cancelled in-flight API_VERSIONS request with correlation id 9 due to node -5 being disconnected (elapsed time since creation: 58ms, elapsed time since send: 58ms, request timeout: 30000ms)
答：
0赞
OneCricketeer
11/18/2023
#1
您需要在所有 kafka 值前面加上前缀。请参阅文档。spring.kafka
此外，似乎与.你想要吗？sasl.mechanism=PLAINsecurity.protocol=SASL_SSLSASL_PLAINTEXT
该错误仅表示引导服务器无法连接，并且生产者正在尝试连接到值中的 9 个代理中的每一个 - 对于任何初始连接。broker-101broker-109
这也可能意味着 SASL 配置错误（用户名/密码）。
由于您正在尝试使用 ，这意味着您还需要设置值，例如 、 等。SASL_SSLspring.kafka.properties[ssl.key.password]spring.kafka.properties[ssl.keystore.location]
这也可能意味着您从客户端使用的连接协议的服务器端属性不正确。advertised.listeners
在迁移到 Spring 客户端之前，您应该尝试使用相同的设置来验证连接是否有效。kafka-console-producer
以下是 Confluent 提供的有关为 Confluent Cloud 或本地 Kafka 配置 Spring Boot 的教程。
https://developer.confluent.io/get-started/spring-boot/#introduction
评论
0赞
Sybuser
11/18/2023
“sasl.mechanism=PLAIN 似乎与 security.protocol=SASL_SSL 相矛盾。你想要SASL_PLAINTEXT吗？“这是 kafka 管理员告诉我要使用的，所以我必须提出这是矛盾的。似乎所有这些属性都需要设置？spring.kafka.properties.ssl.truststore.location spring.kafka.properties.ssl.truststore.password spring.kafka.properties.ssl.keystore.location spring.kafka.properties.ssl.keystore.password spring.kafka.properties.ssl.key.password 他们还向我指出了十几个crt和p7c文件，我也可以使用这些文件类型吗？
0赞
OneCricketeer
11/19/2023
在不知道 Kafka 服务器是如何设置的情况下，我无法回答哪些属性是必要的。如果你使用 ssl，那么这些都应该设置，是的。有关文件类型，请参阅 Kafka 文档本身。我认为只允许 JKS、p12 和 crt 文件
评论