提问人:Doul 提问时间:10/27/2023 更新时间:11/3/2023 访问量:78
mongodb-kubernetes-operator - mongodb-agent 永远不会就绪
mongodb-kubernetes-operator - mongodb-agent is never ready
问:
你做了什么来遇到这个错误? 重现行为的步骤:
部署管理器 v0.8.2 然后是 MongoDBCommunity v6.0.5
你期待什么?Mongodb-agent 就绪且集群正常
发生了什么?侧容器 mongodb-agent 永远不会进入就绪状态。
错误:
- 就绪探测失败:命令“/opt/scripts/readinessprobe”超时
- 在mongodbagent日志中:“就绪探测失败:恐慌:打开/var/log/mongodb-mms-automation/healthstatus/agent-health-status.json:没有这样的文件或目录goroutine 1 [running]:main.main()/workspace/cmd/readiness/main.go:226”
- 在具有就绪探测 v1.0.17 而不是 v1.0.15 的 mongodbagent 日志中: cat: /mongodb-automation/agent-api-key/agentApiKey:没有这样的文件或目录
运营商信息运营商版本:0.8.2 使用的MongoDB镜像:6.0.5
Kubernetes 集群信息分布 : tkgi 版本 : 1.24 映像注册表位置(码头或内部注册表):码头 其他背景 我们尝试将 readinessprobe 版本从 1.0.15 升级到 1.0.17,但我们遇到了同样的问题
**MongoDBCommunity YAML:**
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongodb-replicaset
spec:
members: 1
type: ReplicaSet
version: "6.0.5"
security:
authentication:
modes: ["SCRAM"]
users:
- name: user-admin
db: mydb
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: user-admin-mongo-secret
key: MONGO_PASSWORD
roles:
- name: clusterAdmin
db: mydb
- name: dbOwner
db: mydb
scramCredentialsSecretName: my-scram
statefulSet:
spec:
template:
spec:
containers:
- name: mongod
resources:
limits:
cpu: "200m"
memory: 500M
requests:
cpu: "50m"
memory: 100M
- name: mongodb-agent
resources:
limits:
cpu: "200m"
memory: 250M
requests:
cpu: "50m"
memory: 50M
additionalMongodConfig:
storage.wiredTiger.engineConfig.journalCompressor: zlib
操作员的日志文件
Running ./manager
2023-10-26T14:36:16.874Z INFO manager/main.go:74 Watching namespace: my-namespace
I1026 14:36:17.925053 8 request.go:690] Waited for 1.03553922s due to client-side throttling, not priority and fairness, request: GET:https://56.0.0.1:443/apis/tap.linkerd.io/v1alpha1?timeout=32s
2023-10-26T14:36:20.635Z INFO manager/main.go:91 Registering Components.
2023-10-26T14:36:20.635Z INFO manager/main.go:104 Starting the Cmd.
2023-10-26T14:36:20.737Z INFO controllers/replica_set_controller.go:139 Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.737Z DEBUG controllers/replica_set_controller.go:141 Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.737Z DEBUG controllers/replica_set_controller.go:151 Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.838Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.939Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.041Z INFO controllers/replica_set_controller.go:470 Create/Update operation succeeded {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:21.041Z INFO controllers/replica_set_controller.go:362 Creating/Updating AutomationConfig {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z DEBUG scram/scram.go:172 Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:21.055Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z DEBUG agent/replica_set_port_manager.go:40 Calculated process port map: map[ds042-mongodb-replicaset-0:27017] {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z DEBUG controllers/replica_set_controller.go:537 AutomationConfigMembersThisReconciliation {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:21.055Z DEBUG controllers/replica_set_controller.go:385 Waiting for agents to reach version 3 {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z INFO controllers/mongodb_status_options.go:110 ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:21.063Z INFO controllers/replica_set_controller.go:139 Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z DEBUG controllers/replica_set_controller.go:141 Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z DEBUG controllers/replica_set_controller.go:151 Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.167Z INFO controllers/replica_set_controller.go:470 Create/Update operation succeeded {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:21.167Z INFO controllers/replica_set_controller.go:362 Creating/Updating AutomationConfig {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z DEBUG scram/scram.go:172 Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:21.180Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z DEBUG agent/replica_set_port_manager.go:40 Calculated process port map: map[ds042-mongodb-replicaset-0:27017] {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z DEBUG controllers/replica_set_controller.go:537 AutomationConfigMembersThisReconciliation {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:21.180Z DEBUG controllers/replica_set_controller.go:385 Waiting for agents to reach version 3 {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z INFO controllers/mongodb_status_options.go:110 ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:31.064Z INFO controllers/replica_set_controller.go:139 Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z DEBUG controllers/replica_set_controller.go:141 Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z DEBUG controllers/replica_set_controller.go:151 Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.181Z INFO controllers/replica_set_controller.go:470 Create/Update operation succeeded {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:31.181Z INFO controllers/replica_set_controller.go:362 Creating/Updating AutomationConfig {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z DEBUG scram/scram.go:172 Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:31.194Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z DEBUG agent/replica_set_port_manager.go:40 Calculated process port map: map[ds042-mongodb-replicaset-0:27017] {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z DEBUG controllers/replica_set_controller.go:537 AutomationConfigMembersThisReconciliation {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:31.195Z DEBUG controllers/replica_set_controller.go:385 Waiting for agents to reach version 3 {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.195Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.195Z INFO controllers/mongodb_status_options.go:110 ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:38.234Z INFO controllers/replica_set_controller.go:139 Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.234Z DEBUG controllers/replica_set_controller.go:141 Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z DEBUG controllers/replica_set_controller.go:151 Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.359Z INFO controllers/replica_set_controller.go:470 Create/Update operation succeeded {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:38.359Z INFO controllers/replica_set_controller.go:362 Creating/Updating AutomationConfig {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z DEBUG scram/scram.go:172 Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:38.371Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z DEBUG agent/replica_set_port_manager.go:122 No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z DEBUG agent/replica_set_port_manager.go:40 Calculated process port map: map[ds042-mongodb-replicaset-0:27017] {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z DEBUG controllers/replica_set_controller.go:537 AutomationConfigMembersThisReconciliation {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:38.371Z DEBUG controllers/replica_set_controller.go:385 Waiting for agents to reach version 3 {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z INFO controllers/mongodb_status_options.go:110 ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:41.203Z INFO controllers/replica_set_controller.go:139 Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z DEBUG controllers/replica_set_controller.go:141 Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z DEBUG controllers/replica_set_controller.go:151 Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z DEBUG agent/agent_readiness.go:106 The Pod 'ds042-mongodb-replicaset-0' doesn't have
答:
@Doul 从您描述的行为来看,您正在将其安装在 arm64 机器上运行的集群中,就像我在配备 M1-M2 芯片的新 Mac 上一样。我已经为此发疯了一段时间。
尽管 operator 0.8.3 已与 arm64 兼容,但仍然存在问题。
它指向的 mongo-agent 还没有正式的 arm64 映像 ( https://quay.io/repository/mongodb/mongodb-agent?tab=tags),而 operator、version-upgrade-post-start-hook 和 readinessprobe 有,因此当通过 operator 安装 mongo 时(operator 指向 quay.io 映像),安装的 mongo-agent 与 arm64 架构芯片不兼容,并且 mongo pod 与事件一样挂起,因为 mongod 容器确实被创建,但代理没有。
Dough 代理的官方 arm64 映像尚不可用,二进制文件可用,值得庆幸的是,它们已由 GitHub 上的 @mohsinonxrm 构建到 operator 8.0.1,并在 DockerHub 上可用,请参阅此处 https://github.com/mongodb/mongodb-kubernetes-operator/issues/299#issuecomment-1666717878。
因此,只需通过创建一个 value.yaml(根据需要命名)文件并安装带有标志的图表,覆盖运算符图表值以使用这些图像而不是官方 quay.io。READY 1/2
Readiness probe failed: command "/opt/scripts/readinessprobe" timed out
-f path/to/the/values.yaml
值.yaml
operator:
name: mongodb-kubernetes-operator
operatorImageName: mongodb-kubernetes-operator
deploymentName: mongodb-kubernetes-operator
version: 0.8.1
resources:
limits:
cpu: 200m
memory: 200Mi
requests:
cpu: 200m
memory: 200Mi
agent:
name: mongodb-agent
version: 12.0.24.7719-1
versionUpgradeHook:
name: mongodb-kubernetes-operator-version-upgrade-post-start-hook
version: 1.0.7
readinessProbe:
name: mongodb-kubernetes-readiness
version: 1.0.15
registry:
agent: docker.io/mohsinonxrm #quay.io/mongodb
versionUpgradeHook: docker.io/mohsinonxrm #quay.io/mongodb
readinessProbe: docker.io/mohsinonxrm #quay.io/mongodb
operator: docker.io/mohsinonxrm #quay.io/mongodb
有了这个,我在我的 M6.0.11 Mac 上的 k3d 集群上运行 mongo 1。 希望 mongo-agent 的官方 arm64 映像将很快发布,以便使用官方操作员 0.8.3。
评论