在 Kubernetes docker-compose 中创建 CRON 作业之间的依赖关系-解网

问：

在我当前的项目中，Kubernetes 中安排了多个 CRON 作业。每个 cron 作业都计划了特定的开始时间。但是，根据新的要求，我们需要在作业之间创建依赖关系，以便作业不应等到计划时间，而是应该在上一个作业完成时立即启动。

例如，假设我们有 4 个 CRON 作业（a1、a2、a3、a4）计划在下午 1 点、2 点、3 点和 4 点开始。但是假设 a2 在 10 分钟内完成（下午 2 点开始，下午 2 点 10 分结束）。A3 而不是等到下午 3 点，它可以在下午 2 点 10 分触发吗？如果是，我们如何实现这一目标。

我当前的docker-compose.kubernetes.yml文件如下所示 -

version: "3.9"

services:
  purelit-run-migrations:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 9 1 1 *"
    deploy:
      resources:
        limits:
          memory: "300M"
          cpus: '0.4'
        reservations:
          cpus: '0.4'
          memory: "300M"
    command: ["/app/purelit/bin/python" ,"/app/utils/init_schema.py", "--script_start_index","105", "--script_end_index","105"]

  a1-purelit-copy-sftp-files:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 8 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.0'
        reservations:
          cpus: '1.0'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/utils/copy_files.py"]

  a2-purelit-supply-loader:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 9 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.0'
        reservations:
          cpus: '1.0'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/loader/supply_data/supply_data_pipeline_runner.py"]

  a3-purelit-supply-delta-export:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 10 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.2'
        reservations:
          cpus: '1.2'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/delta/export/etl_supply_to_app_data_sync_pipeline_runner.py", "--source","all"]

在上面的代码中我们可以做哪些配置来实现CRON作业依赖？

如果不是CRON工作，那么如何实现它。

docker kubernetes cron job-scheduling kubernetes-cronjob

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cron-job
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: job-service-account
          containers:
          - name: cron-container
            image: your-cron-image:tag
            command: ["sh", "-c", "your-cron-command && kubectl create job --from=cronjob/cron-job some-job"]

其他工作（或工作）：

apiVersion: batch/v1
kind: Job
metadata:
  name: some-job
spec:
  template:
    spec:
      serviceAccountName: job-service-account
      containers:
      - name: some-container
        image: your-image:tag
        command: ["your-command"]

RBAC（需要允许 cronjob 创建其他作业）：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: job-service-account

---

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: job-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "create"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: job-role-binding
subjects:
- kind: ServiceAccount
  name: job-service-account
  namespace: default
roleRef:
  kind: Role
  name: job-role
  apiGroup: rbac.authorization.k8s.io

在 Kubernetes docker-compose 中创建 CRON 作业之间的依赖关系

Create dependency between CRON jobs in Kubernetes docker-compose

评论

评论