mongodb-kubernetes-operator - mongodb-agent 永远不会就绪

mongodb-kubernetes-operator - mongodb-agent is never ready

提问人:Doul 提问时间:10/27/2023 更新时间:11/3/2023 访问量:78

问:

你做了什么来遇到这个错误? 重现行为的步骤:

部署管理器 v0.8.2 然后是 MongoDBCommunity v6.0.5

你期待什么?Mongodb-agent 就绪且集群正常

发生了什么?侧容器 mongodb-agent 永远不会进入就绪状态。

错误:

  • 就绪探测失败:命令“/opt/scripts/readinessprobe”超时
  • 在mongodbagent日志中:“就绪探测失败:恐慌:打开/var/log/mongodb-mms-automation/healthstatus/agent-health-status.json:没有这样的文件或目录goroutine 1 [running]:main.main()/workspace/cmd/readiness/main.go:226”
  • 在具有就绪探测 v1.0.17 而不是 v1.0.15 的 mongodbagent 日志中: cat: /mongodb-automation/agent-api-key/agentApiKey:没有这样的文件或目录

运营商信息运营商版本:0.8.2 使用的MongoDB镜像:6.0.5

Kubernetes 集群信息分布 : tkgi 版本 : 1.24 映像注册表位置(码头或内部注册表):码头 其他背景 我们尝试将 readinessprobe 版本从 1.0.15 升级到 1.0.17,但我们遇到了同样的问题

**MongoDBCommunity YAML:**

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: mongodb-replicaset
spec:
  members: 1
  type: ReplicaSet
  version: "6.0.5"
  security:
    authentication:
      modes: ["SCRAM"]
  users:
    - name: user-admin
      db: mydb
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password
        name: user-admin-mongo-secret
        key: MONGO_PASSWORD
      roles:
        - name: clusterAdmin
          db: mydb
        - name: dbOwner
          db: mydb
      scramCredentialsSecretName: my-scram
  statefulSet:
    spec:
      template:
        spec:
          containers:
            - name: mongod
              resources:
                limits:
                  cpu: "200m"
                  memory: 500M
                requests:
                  cpu: "50m"
                  memory: 100M
            - name: mongodb-agent
              resources:
                limits:
                  cpu: "200m"
                  memory: 250M
                requests:
                  cpu: "50m"
                  memory: 50M
  additionalMongodConfig:
    storage.wiredTiger.engineConfig.journalCompressor: zlib

操作员的日志文件

Running ./manager
2023-10-26T14:36:16.874Z    INFO    manager/main.go:74  Watching namespace: my-namespace
I1026 14:36:17.925053       8 request.go:690] Waited for 1.03553922s due to client-side throttling, not priority and fairness, request: GET:https://56.0.0.1:443/apis/tap.linkerd.io/v1alpha1?timeout=32s
2023-10-26T14:36:20.635Z    INFO    manager/main.go:91  Registering Components.
2023-10-26T14:36:20.635Z    INFO    manager/main.go:104 Starting the Cmd.
2023-10-26T14:36:20.737Z    INFO    controllers/replica_set_controller.go:139   Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.737Z    DEBUG   controllers/replica_set_controller.go:141   Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.737Z    DEBUG   controllers/replica_set_controller.go:151   Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.838Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:20.939Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.041Z    INFO    controllers/replica_set_controller.go:470   Create/Update operation succeeded   {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:21.041Z    INFO    controllers/replica_set_controller.go:362   Creating/Updating AutomationConfig  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    DEBUG   scram/scram.go:172  Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:21.055Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    DEBUG   agent/replica_set_port_manager.go:40    Calculated process port map: map[ds042-mongodb-replicaset-0:27017]  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    DEBUG   controllers/replica_set_controller.go:537   AutomationConfigMembersThisReconciliation   {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:21.055Z    DEBUG   controllers/replica_set_controller.go:385   Waiting for agents to reach version 3   {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.055Z    INFO    controllers/mongodb_status_options.go:110   ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:21.063Z    INFO    controllers/replica_set_controller.go:139   Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z    DEBUG   controllers/replica_set_controller.go:141   Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z    DEBUG   controllers/replica_set_controller.go:151   Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.063Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.167Z    INFO    controllers/replica_set_controller.go:470   Create/Update operation succeeded   {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:21.167Z    INFO    controllers/replica_set_controller.go:362   Creating/Updating AutomationConfig  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    DEBUG   scram/scram.go:172  Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:21.180Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    DEBUG   agent/replica_set_port_manager.go:40    Calculated process port map: map[ds042-mongodb-replicaset-0:27017]  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    DEBUG   controllers/replica_set_controller.go:537   AutomationConfigMembersThisReconciliation   {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:21.180Z    DEBUG   controllers/replica_set_controller.go:385   Waiting for agents to reach version 3   {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:21.180Z    INFO    controllers/mongodb_status_options.go:110   ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:31.064Z    INFO    controllers/replica_set_controller.go:139   Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z    DEBUG   controllers/replica_set_controller.go:141   Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z    DEBUG   controllers/replica_set_controller.go:151   Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.064Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.181Z    INFO    controllers/replica_set_controller.go:470   Create/Update operation succeeded   {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:31.181Z    INFO    controllers/replica_set_controller.go:362   Creating/Updating AutomationConfig  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z    DEBUG   scram/scram.go:172  Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:31.194Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z    DEBUG   agent/replica_set_port_manager.go:40    Calculated process port map: map[ds042-mongodb-replicaset-0:27017]  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.194Z    DEBUG   controllers/replica_set_controller.go:537   AutomationConfigMembersThisReconciliation   {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:31.195Z    DEBUG   controllers/replica_set_controller.go:385   Waiting for agents to reach version 3   {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.195Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:31.195Z    INFO    controllers/mongodb_status_options.go:110   ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:38.234Z    INFO    controllers/replica_set_controller.go:139   Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.234Z    DEBUG   controllers/replica_set_controller.go:141   Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z    DEBUG   controllers/replica_set_controller.go:151   Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.235Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.359Z    INFO    controllers/replica_set_controller.go:470   Create/Update operation succeeded   {"ReplicaSet": "my-namespace/mongodb-replicaset", "operation": "updated"}
2023-10-26T14:36:38.359Z    INFO    controllers/replica_set_controller.go:362   Creating/Updating AutomationConfig  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    DEBUG   scram/scram.go:172  Credentials have not changed, using credentials stored in: secret/my-scram-scram-credentials
2023-10-26T14:36:38.371Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    DEBUG   agent/replica_set_port_manager.go:122   No port change required {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    DEBUG   agent/replica_set_port_manager.go:40    Calculated process port map: map[ds042-mongodb-replicaset-0:27017]  {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    DEBUG   controllers/replica_set_controller.go:537   AutomationConfigMembersThisReconciliation   {"mdb.AutomationConfigMembersThisReconciliation()": 1}
2023-10-26T14:36:38.371Z    DEBUG   controllers/replica_set_controller.go:385   Waiting for agents to reach version 3   {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have annotation 'agent.mongodb.com/version' yet    {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:38.371Z    INFO    controllers/mongodb_status_options.go:110   ReplicaSet is not yet ready, retrying in 10 seconds
2023-10-26T14:36:41.203Z    INFO    controllers/replica_set_controller.go:139   Reconciling MongoDB {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z    DEBUG   controllers/replica_set_controller.go:141   Validating MongoDB.Spec {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z    DEBUG   controllers/replica_set_controller.go:151   Ensuring the service exists {"ReplicaSet": "my-namespace/mongodb-replicaset"}
2023-10-26T14:36:41.203Z    DEBUG   agent/agent_readiness.go:106    The Pod 'ds042-mongodb-replicaset-0' doesn't have 
MongoDB Kubernetes 运算符

评论


答:

0赞 Vincenzo 11/3/2023 #1

@Doul 从您描述的行为来看,您正在将其安装在 arm64 机器上运行的集群中,就像我在配备 M1-M2 芯片的新 Mac 上一样。我已经为此发疯了一段时间。 尽管 operator 0.8.3 已与 arm64 兼容,但仍然存在问题。 它指向的 mongo-agent 还没有正式的 arm64 映像 ( https://quay.io/repository/mongodb/mongodb-agent?tab=tags),而 operator、version-upgrade-post-start-hook 和 readinessprobe 有,因此当通过 operator 安装 mongo 时(operator 指向 quay.io 映像),安装的 mongo-agent 与 arm64 架构芯片不兼容,并且 mongo pod 与事件一样挂起,因为 mongod 容器确实被创建,但代理没有。 Dough 代理的官方 arm64 映像尚不可用,二进制文件可用,值得庆幸的是,它们已由 GitHub 上的 @mohsinonxrm 构建到 operator 8.0.1,并在 DockerHub 上可用,请参阅此处 https://github.com/mongodb/mongodb-kubernetes-operator/issues/299#issuecomment-1666717878。 因此,只需通过创建一个 value.yaml(根据需要命名)文件并安装带有标志的图表,覆盖运算符图表值以使用这些图像而不是官方 quay.io。READY 1/2 Readiness probe failed: command "/opt/scripts/readinessprobe" timed out-f path/to/the/values.yaml

值.yaml

operator:
  name: mongodb-kubernetes-operator
  operatorImageName: mongodb-kubernetes-operator
  deploymentName: mongodb-kubernetes-operator
  version: 0.8.1
  resources:
    limits:
      cpu: 200m 
      memory: 200Mi 
    requests:
      cpu: 200m 
      memory: 200Mi 
agent:
  name: mongodb-agent
  version: 12.0.24.7719-1
versionUpgradeHook:
  name: mongodb-kubernetes-operator-version-upgrade-post-start-hook
  version: 1.0.7
readinessProbe:
  name: mongodb-kubernetes-readiness
  version: 1.0.15

registry:
  agent: docker.io/mohsinonxrm #quay.io/mongodb
  versionUpgradeHook: docker.io/mohsinonxrm #quay.io/mongodb
  readinessProbe: docker.io/mohsinonxrm #quay.io/mongodb
  operator: docker.io/mohsinonxrm #quay.io/mongodb

有了这个,我在我的 M6.0.11 Mac 上的 k3d 集群上运行 mongo 1。 希望 mongo-agent 的官方 arm64 映像将很快发布,以便使用官方操作员 0.8.3。